-- Logs begin at Tue 2022-08-09 06:20:09 UTC, end at Tue 2022-08-09 07:28:58 UTC. -- Aug 09 06:48:50.653621 dv-cinder-852514-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 06:48:50.661579 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 06:48:50.663620 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 06:48:50 2022] *** Aug 09 06:48:50.663620 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: nodename: dv-cinder-852514-1-iscsi Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: machine: x86_64 Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: clock source: unix Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: pcre jit disabled Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: detected number of CPU cores: 4 Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: current working directory: / Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: detected binary path: /usr/bin/uwsgi-core Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: your processes number limit is 47680 Aug 09 06:48:50.664639 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: your memory page size is 4096 bytes Aug 09 06:48:50.665168 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: detected max file descriptor number: 2048 Aug 09 06:48:50.665168 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: lock engine: pthread robust mutexes Aug 09 06:48:50.665168 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: thunder lock: enabled Aug 09 06:48:50.665168 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 06:48:50.665168 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 06:48:50.704505 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: Python main interpreter initialized at 0x558ec16529c0 Aug 09 06:48:50.704505 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: python threads support enabled Aug 09 06:48:50.704505 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: your server socket listen backlog is limited to 100 connections Aug 09 06:48:50.704505 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: your mercy for graceful operations on workers is 90 seconds Aug 09 06:48:50.704881 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 06:48:50.704951 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: *** Operational MODE: preforking *** Aug 09 06:48:50.705192 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: *** uWSGI is running in multiple interpreter mode *** Aug 09 06:48:50.705192 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: spawned uWSGI master process (pid: 80160) Aug 09 06:48:50.705393 dv-cinder-852514-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 06:48:50.705942 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: spawned uWSGI worker 1 (pid: 80168, cores: 1) Aug 09 06:48:50.707076 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: spawned uWSGI worker 2 (pid: 80169, cores: 1) Aug 09 06:48:50.707076 dv-cinder-852514-1-iscsi devstack@keystone.service[80160]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 06:48:56.223228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 06:48:56.223228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes Aug 09 06:48:56.224080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 06:48:56.224080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Aug 09 06:48:56.224499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 06:48:56.224499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes Aug 09 06:48:56.224608 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 06:48:56.224608 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Aug 09 06:48:56.274563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 06:48:56.274740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 06:48:56.274940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] command line args: [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 06:48:56.275216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 06:48:56.275476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 06:48:56.275570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ================================================================================ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 06:48:56.275744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 06:48:56.275824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.275893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 06:48:56.276071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_dir = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.276153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 06:48:56.276219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.276363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 06:48:56.276454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_source = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.276583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] debug = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.276660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.276791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.281128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.281128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.281814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.282028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.282572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.282904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.283225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.283449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.284029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.284261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.284261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.284605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.284605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_config_append = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.285190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.285537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.285707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.285933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_dir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.286218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.286447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.286612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.286780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.286945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.287768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.288955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.289241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.289376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.289538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.289993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.290202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.290341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.290473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.291093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.291230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.291357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.291528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.291728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.292282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.292410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.292534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.292701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.292701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.293125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.293125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] publish_errors = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.293457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.293457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.293767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.293900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.294939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.295065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] standard_threads = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.295189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.295189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.296575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.296783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.296911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_journal = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_json = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.297918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.298045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_stderr = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.298170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.298302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_syslog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.298435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.298567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 06:48:56.298700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.298957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.299140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.299547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.300467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.300467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.301464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.301574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.302580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.302768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.302849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.303918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.304956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.305882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.306906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.307930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.308996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.309998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.310574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.312953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.313980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.314963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.315906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.316932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.317923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.318938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.319929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.320958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.external = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.321999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.password = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.token = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.322933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.323965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.324984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.325958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.326457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.327964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.328996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.329984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.330994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.331990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332642 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.332930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.333949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.334984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.335985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.336953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.337927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.338877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.339902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.340852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.341987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.342062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.342895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.343911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.344876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.345867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346273 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.346834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.347997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.348989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.349975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.350944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.351989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.352935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.353931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.354916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355074 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.355997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.356845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.357811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.358994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.359828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.360839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.361994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.362968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.363991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364321 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.364980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.365871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 06:48:56.365948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.366172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.366413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.366662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.366903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.367156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.367398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.367674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.367913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.368160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.368401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 06:48:56.368578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 06:48:56.396093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.396329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.396561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.397808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.398043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.398272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.407274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.408933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.435766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.436147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.437035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.437266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.489372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.489657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.540804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.541039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.541157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.541396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.606712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.606939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.607996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.608203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.631758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.631918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.632904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.633040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.633119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.633257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.644889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.644990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.657778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.657892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.658996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.659127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.659206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 06:48:56.676207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.676401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.676485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 06:48:56.676617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 06:48:56.676756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.676964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.677065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 06:48:56.677169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 06:48:56.677250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.677442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.677520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.677703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.677784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.677962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.678041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.678222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.690990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.697868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.697961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.700976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.701072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.701268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.701349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.701644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.702221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.702459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.702558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.702792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.702871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.703992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.704172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.704252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.704443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.704522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.727335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.727766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.728861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.729150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.729275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.729566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.733816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.734098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.734367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.734449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.734659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.734949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.741057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.741307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.741616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.741911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.745414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.745709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.746293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.746955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.747152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.747241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.747424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.747511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.747755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.748043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.764695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.764979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.765604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.765891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.768984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.769888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.772168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.772467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.773093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.773394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.776856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.777839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.780656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.781645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.783925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.784229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.784523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.784810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.784891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.785101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.785182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.785391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.785474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.785700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.785981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.786267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.801966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.802256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.802525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.802791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.802990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.803286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.803568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.803867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.809754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.810944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.811520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.811918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.812559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.812843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.814743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.815032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.815332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.815652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.815735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.815938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.816032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.816280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.816585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.816876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.827414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.827729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.828015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.828298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.828449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.828585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.828739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.828864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.829028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.829124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.829319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.829609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.829894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.830177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.846569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.847416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.849593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.850448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.852560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.853441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.854132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.855054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.855654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.855944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.856242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.856593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.856899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.857188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.863380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.864368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.865028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.866007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.867991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.868288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.868578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.868864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.869005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.869303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.869603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.869892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.877299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.877597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.877946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.878455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.878755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.879046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.884870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.885164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.885450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.885731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.885971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 06:48:56.886063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.886265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.886357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.886545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.886646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.886817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.886919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.887986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.888078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.888293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.888383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.888559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.888650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.888912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.889320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.889616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.889893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 06:48:56.923483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558ec16529c0 pid: 80168 (default app) Aug 09 06:48:56.924375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558ec16529c0 pid: 80169 (default app) Aug 09 06:48:56.926817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781f032-bb18-43e7-9322-10e89395d680 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:48:56.927105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781f032-bb18-43e7-9322-10e89395d680 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:48:56.927377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781f032-bb18-43e7-9322-10e89395d680 None None] PATH_INFO: `/v3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:48:56.931652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Tue Aug 9 06:48:56 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:04.642695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:04.643258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:04.643258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:04.647670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:04.652872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:04.653302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:04.653419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:04.724165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: INFO dbcounter [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] Registered counter for database keystone Aug 09 06:49:04.742967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG oslo_db.sqlalchemy.engines [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80168) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 06:49:04.749461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writer thread running {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 06:49:04.927931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80168) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 06:49:04.928262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80168) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 06:49:04.982538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:04.984280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:05.112997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 06:49:05.270649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:05.340082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] Registered counter for database keystone Aug 09 06:49:05.361454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 06:49:05.367572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 06:49:05.487457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87949d-fdf9-4686-8b01-39db4bb84d71 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:05.491727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:05.492349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:05.492632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:05.492909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:05.502501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:05.555738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 06:49:05.556030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bc638-6624-4965-9cee-e48fd7b534ad admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:05.566395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Aug 9 06:49:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 06:49:05.572333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:05.586743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55892c-1c59-4180-a8da-49461ea4bf61 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:05.591151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:05.593012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:05.593294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:05.593568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] PATH_INFO: `/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:05.595015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] RBAC: Authorizing `identity:get_project(project_id=489e2c3ffdae42c6a83d2b2655912ea9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:05.645568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING keystone.common.rbac_enforcer.enforcer [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 06:49:05.645909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521166d5-56bc-44be-927e-1af3fd35522d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:05.650421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Tue Aug 9 06:49:05 2022] GET /identity/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:07.621164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9882113-9d74-414e-a014-c3b6abb3db39 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:07.621707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9882113-9d74-414e-a014-c3b6abb3db39 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:07.621707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9882113-9d74-414e-a014-c3b6abb3db39 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:07.626091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:07.631583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:07.631958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:07.632169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:07.653577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:07.655799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:07.663619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 06:49:07.690254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9882113-9d74-414e-a014-c3b6abb3db39 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:07.703293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9882113-9d74-414e-a014-c3b6abb3db39 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:07.707549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c83f7b-7c93-403d-89a8-5dfc18366e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:07.708176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c83f7b-7c93-403d-89a8-5dfc18366e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:07.708461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c83f7b-7c93-403d-89a8-5dfc18366e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:07.708737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c83f7b-7c93-403d-89a8-5dfc18366e86 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:07.716349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c83f7b-7c93-403d-89a8-5dfc18366e86 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:07.718828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c83f7b-7c93-403d-89a8-5dfc18366e86 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:07.727520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Aug 9 06:49:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 06:49:07.733661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:07.742582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61776611-f68f-4d7e-a3be-c20fd154015e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:07.746942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94270-ba0b-4568-b12a-76a208c0a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:07.747561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94270-ba0b-4568-b12a-76a208c0a3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:07.747877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94270-ba0b-4568-b12a-76a208c0a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:07.748150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94270-ba0b-4568-b12a-76a208c0a3aa admin admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:07.749552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94270-ba0b-4568-b12a-76a208c0a3aa admin admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:07.751353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94270-ba0b-4568-b12a-76a208c0a3aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:07.753133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Tue Aug 9 06:49:07 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:09.788941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc8a2-49fd-4367-bed3-3101ea12ee43 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:09.789478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc8a2-49fd-4367-bed3-3101ea12ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:09.789478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc8a2-49fd-4367-bed3-3101ea12ee43 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:09.790850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:09.796498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:09.796906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:09.797126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:09.815549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:09.816756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:09.828672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05b328-1f94-496f-a5f1-797a3310d570 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:09.828955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05b328-1f94-496f-a5f1-797a3310d570 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:09.829231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05b328-1f94-496f-a5f1-797a3310d570 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:09.830541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:09.835704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:09.835994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:09.836273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:09.880503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 06:49:09.880936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 06:49:09.899433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:09.922656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:09.926236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:09.927863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:09.946205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.027359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4cb90-c0e0-4f5f-b656-1c4a15876cac None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.028536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:10.050786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.061533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.062177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.062455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.062735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.073869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:10.082639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:10.083765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:10.084955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:10.089503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.152970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ac95-6acb-4024-b3be-f1885e0fc9cf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.155854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.172916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.191523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.191810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa36072-8390-40dd-9896-e14487051c25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.192420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.192703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.192977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.225575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.226176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.226460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.226738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.227413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:10.227745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.229139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.241493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:10.246788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.261907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.263514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.367044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/16] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:49:10.373098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.378190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.457589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbeb2-1f8a-4ae2-805e-931128ccaa7e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.479834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a388e86-2368-464d-b637-d10fc65c8a7f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.491865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.492496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.492778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.493060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.494184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.495656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.497209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:10.503406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.514603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.515457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.515770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.516041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.518064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.533966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.535416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:49:10.554434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:10.555297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:49:10.556295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:49:10.556295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:49:10.556295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:10.556295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:10.561908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.604638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb94c3-6a73-4d6e-a099-860d83f837db None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.618844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.641007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.641636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.642158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.642158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.643278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.644747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.646347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:10.651918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:10.662816 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3733a-6238-4bfc-8a60-0a6275383530 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.696536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.697138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.697417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.697686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.698320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:10.698612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.700024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf265b-c927-44aa-ba09-8e8916f6b60d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.716691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:10.736645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:10.758107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed744-082d-499a-ab80-6354c7a2fdf5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:10.794392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:10.795023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:10.795500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:10.795887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:10.796643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.domain.id=default, user.id=b26927358e664fe3981d02119bbe804c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:10.797197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:10.798428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5322328-763a-459c-91c7-1776e31e6cd7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:10.805969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 11/21] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Aug 9 06:49:10 2022] GET /identity/v3/role_assignments?user.id=b26927358e664fe3981d02119bbe804c&scope.domain.id=default&role.id=d58e69f2b822466daca66a9696521a24 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:13.098745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ffe2a-a6f6-4325-8448-b54f62fe48d7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.099238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ffe2a-a6f6-4325-8448-b54f62fe48d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.099308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ffe2a-a6f6-4325-8448-b54f62fe48d7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.100638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:13.106049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.106885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.107351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.133705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd99c1b-ed16-439c-9e77-83c7213c6bb0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.133972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd99c1b-ed16-439c-9e77-83c7213c6bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.134162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd99c1b-ed16-439c-9e77-83c7213c6bb0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.135705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/23] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:13.136348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:13.137539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:13.144895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56d0c-4873-4191-8786-7397de174841 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.145174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56d0c-4873-4191-8786-7397de174841 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.145448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56d0c-4873-4191-8786-7397de174841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.184641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bec56d0c-4873-4191-8786-7397de174841 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:13.186441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bec56d0c-4873-4191-8786-7397de174841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:13.237670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 12/24] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:13.296966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:13.395933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/25] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:13.433448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:13.448916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56d0c-4873-4191-8786-7397de174841 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:13.468080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da292de6-76be-45a2-8d2d-a1b313a07fcf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:13.509548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:13.510564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.511226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.512767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.537856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:13.542151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:13.557528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-509a72c5-fe1b-4a6e-8902-5563dd5c2430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:49:13.572133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:13.573013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:49:13.573822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:49:13.573822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:49:13.573822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:13.578037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 13/26] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:13.583282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f03f14-7521-4f51-9f02-bb9ba0d50419 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.585080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f03f14-7521-4f51-9f02-bb9ba0d50419 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.586242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f03f14-7521-4f51-9f02-bb9ba0d50419 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.588463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 14/27] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:13.589556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f03f14-7521-4f51-9f02-bb9ba0d50419 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:13.672071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56d0c-4873-4191-8786-7397de174841 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:13.707275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69927dd-6358-4d2c-b8eb-7582712f230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:13.707900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69927dd-6358-4d2c-b8eb-7582712f230a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.708166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69927dd-6358-4d2c-b8eb-7582712f230a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.708441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69927dd-6358-4d2c-b8eb-7582712f230a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.709120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69927dd-6358-4d2c-b8eb-7582712f230a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:13.710503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69927dd-6358-4d2c-b8eb-7582712f230a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:13.714772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f03f14-7521-4f51-9f02-bb9ba0d50419 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:13.744018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:13.754286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:13.755314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.755868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.756361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.757278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:13.757802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:13.759619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3426d37-c678-43f0-9733-030d11b563a4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:13.783435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 15/28] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:13.785162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f62c95-5b7d-4571-9bff-9de67b14da42 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.785656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f62c95-5b7d-4571-9bff-9de67b14da42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.786130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f62c95-5b7d-4571-9bff-9de67b14da42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.806717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-97f62c95-5b7d-4571-9bff-9de67b14da42 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:13.808377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97f62c95-5b7d-4571-9bff-9de67b14da42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:13.888469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 16/29] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:13.891629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be4936-ca72-4467-ab3a-23e35e0d1d77 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.892142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be4936-ca72-4467-ab3a-23e35e0d1d77 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.892630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be4936-ca72-4467-ab3a-23e35e0d1d77 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.894128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 17/30] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:13.895010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be4936-ca72-4467-ab3a-23e35e0d1d77 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:13.955206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/31] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 9 06:49:13 2022] POST /identity/v3/roles => generated 219 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:49:13.956713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:13.957001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:13.957280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:13.977221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:13.978941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:14.016057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be4936-ca72-4467-ab3a-23e35e0d1d77 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:14.036197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:14.057619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:14.058597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.059076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.059633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.060554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:14.063003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:14.065175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/32] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:14.065862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:14.136842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:14.214098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a73237-1a86-4ccc-9626-a0c10a9563a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:14.254198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f63385-f86e-45f5-b46a-821d9f155885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:14.254923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f63385-f86e-45f5-b46a-821d9f155885 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.255215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f63385-f86e-45f5-b46a-821d9f155885 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.255495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f63385-f86e-45f5-b46a-821d9f155885 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.256196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f63385-f86e-45f5-b46a-821d9f155885 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:14.257764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f63385-f86e-45f5-b46a-821d9f155885 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:14.260298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 18/33] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:49:13 2022] POST /identity/v3/projects => generated 301 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:14.261896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:14.276014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/34] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 9 06:49:14 2022] POST /identity/v3/roles => generated 225 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:14.386857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d277ae-0728-4dc1-ac57-5e45fcfe1c80 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:14.410465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d107-7456-4e34-80c7-f7c9db861a21 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.410792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d107-7456-4e34-80c7-f7c9db861a21 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.411056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d107-7456-4e34-80c7-f7c9db861a21 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.412682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/35] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:14.427384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.427704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.427986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.430241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:14.431091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.431624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.432071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.460591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:14.466204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:14.467539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:14.468707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:14.473939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:14.475418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:14.476896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 19/36] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:14.485861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:14.533432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:14.583470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/37] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:14.608845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:14.627791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1625fb-b71d-48a6-a99a-49e40c43818f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:14.645050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:14.667019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:14.667665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.667957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.668242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.668922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:14.669230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:14.671615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:14.680538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 20/38] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:14.685805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:14.725537 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56d90d-e0f3-45df-93e2-562d0f856a97 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:14.763163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:14.764231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.764725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.765196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.766173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:14.767658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:14.783545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:14.785796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/39] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 9 06:49:14 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:49:14.793676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3c60-0f5c-46a7-8850-1bd57d7ca902 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:14.831731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada5b60-82f1-446d-82af-49c1cd5689f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:14.832387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada5b60-82f1-446d-82af-49c1cd5689f3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.832676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada5b60-82f1-446d-82af-49c1cd5689f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.832954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada5b60-82f1-446d-82af-49c1cd5689f3 None admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.834278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada5b60-82f1-446d-82af-49c1cd5689f3 None admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:14.842647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada5b60-82f1-446d-82af-49c1cd5689f3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:14.842647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 21/40] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:14 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:14.844771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:14.923991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873e81-714c-45d6-b387-f45abe667383 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.924340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873e81-714c-45d6-b387-f45abe667383 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.924561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873e81-714c-45d6-b387-f45abe667383 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.926087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 22/41] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:14.931539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:14.931857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:14.932146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:14.953417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:14.954978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:14.955164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:14.974913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57608-9b6a-482a-bd06-b046ea91feb9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:15.013309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:15.014386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:15.014668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:15.014945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:15.016361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:15.017810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:15.019379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/42] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:15.024975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:15.036766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 23/43] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:15.059479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:15.124729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7afd56-bd7d-40a4-bc1d-960ef252215a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:15.144992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:15.157535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5183a6-4ff9-41e1-be4b-496b9b9c3ebb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:15.159392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:15.160096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:15.160386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:15.160673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] PATH_INFO: `/v3/domains/default/users/b26927358e664fe3981d02119bbe804c/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:15.162862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b26927358e664fe3981d02119bbe804c, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:15.164339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:15.180718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/44] 10.222.0.30 () {58 vars in 1361 bytes} [Tue Aug 9 06:49:15 2022] PUT /identity/v3/domains/default/users/b26927358e664fe3981d02119bbe804c/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:49:15.191911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1bde2-e7b3-4960-b551-1ee828e7f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:15.192602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1bde2-e7b3-4960-b551-1ee828e7f6cd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:15.192887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1bde2-e7b3-4960-b551-1ee828e7f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:15.193243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1bde2-e7b3-4960-b551-1ee828e7f6cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:15.194462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1bde2-e7b3-4960-b551-1ee828e7f6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:15.195994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1bde2-e7b3-4960-b551-1ee828e7f6cd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:15.197856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 24/45] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:15.204443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:15.221827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:15.310612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66101b-87ef-46e3-9861-294418303167 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:15.348237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2c1c-609a-4168-9734-d98fb037e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:15.348860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2c1c-609a-4168-9734-d98fb037e40b None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:15.349146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2c1c-609a-4168-9734-d98fb037e40b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:15.349422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2c1c-609a-4168-9734-d98fb037e40b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:15.350070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2c1c-609a-4168-9734-d98fb037e40b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:15.351535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2c1c-609a-4168-9734-d98fb037e40b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:15.381401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/46] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:49:15 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:17.218214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee77dc-309c-4c62-be9a-51faf7bc0a05 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.218762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee77dc-309c-4c62-be9a-51faf7bc0a05 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.218762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee77dc-309c-4c62-be9a-51faf7bc0a05 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.220144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 25/47] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:17.225210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed27629-2d12-4e33-a79b-42f3754f2fe2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.225600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed27629-2d12-4e33-a79b-42f3754f2fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.225770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed27629-2d12-4e33-a79b-42f3754f2fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.244463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ed27629-2d12-4e33-a79b-42f3754f2fe2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:17.245651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ed27629-2d12-4e33-a79b-42f3754f2fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:17.305043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:17.314351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/48] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:17.336818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee77dc-309c-4c62-be9a-51faf7bc0a05 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:17.395841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ccc9d-5a8d-499a-98e9-52c988e7a272 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.396135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ccc9d-5a8d-499a-98e9-52c988e7a272 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.396415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ccc9d-5a8d-499a-98e9-52c988e7a272 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.397758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/49] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:17.402658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.402949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.403230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.421303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:17.422509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:17.441068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee77dc-309c-4c62-be9a-51faf7bc0a05 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:17.475784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:17.476353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.476636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.476909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.487287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:17.488257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/50] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:17.492101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:17.492993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:17.493715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 26/51] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:17.499969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:17.517057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:17.535758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:17.599337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab850c-ee5c-4b7b-8ead-151b03965932 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:17.619753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b4a4c-c594-4ee9-9f51-7e2bb392f8f1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:17.626434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:17.635883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:17.636504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.636789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.637096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.637771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:17.638076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:17.639688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:17.652229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/52] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:17.653213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:17.653844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.654126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.654955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.656138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:17.657571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:17.658547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:17.659121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 27/53] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:17.664634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:17.748931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:17.758908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac78835-ca4e-481b-8272-e6675c2088fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:17.765016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc819db-0c08-46c1-8a7d-a7fc28f5b72f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:17.791906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:17.792548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.792834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.793110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.794251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:17.796329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:17.797916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/54] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:17 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:17.798905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:17.799572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.799887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.800167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.800812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:17.802342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:17.803302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:17.829467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 28/55] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Aug 9 06:49:17 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:17.864353 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:17.912075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419f1d-7d92-498e-bf6a-2f663f5a185f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:17.948156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55430c5-c282-4dee-b10b-996db15aaebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:17.948810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55430c5-c282-4dee-b10b-996db15aaebb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:17.949093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55430c5-c282-4dee-b10b-996db15aaebb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:17.949368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55430c5-c282-4dee-b10b-996db15aaebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:17.950506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55430c5-c282-4dee-b10b-996db15aaebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:17.952612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55430c5-c282-4dee-b10b-996db15aaebb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:17.954227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/56] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:17.960281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:18.064597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae6d8-c1ec-4200-9af0-5b9e5e5604c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:18.097486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:18.098091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:18.098370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:18.098643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:18.099365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.domain.id=default, user.id=b26927358e664fe3981d02119bbe804c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:18.099701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:18.101150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fe598-07d0-457d-aeba-86d76dfa7c42 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:18.103952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:18.108836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 29/57] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Aug 9 06:49:17 2022] GET /identity/v3/role_assignments?user.id=b26927358e664fe3981d02119bbe804c&scope.domain.id=default&role.id=d58e69f2b822466daca66a9696521a24 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:19.840493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc67f3-8b26-4beb-a203-cd35c13efb15 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:19.840999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc67f3-8b26-4beb-a203-cd35c13efb15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:19.841082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc67f3-8b26-4beb-a203-cd35c13efb15 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:19.842931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/58] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:19.848035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:19.848448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:19.848634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:19.867065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:19.868276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:19.933074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 30/59] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:19.957313 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc67f3-8b26-4beb-a203-cd35c13efb15 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:20.060655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc67f3-8b26-4beb-a203-cd35c13efb15 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:20.084040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:20.096314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a536443-95ba-4112-9369-b2df0407d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:20.097164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a536443-95ba-4112-9369-b2df0407d010 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:20.097596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a536443-95ba-4112-9369-b2df0407d010 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:20.098029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a536443-95ba-4112-9369-b2df0407d010 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:20.099287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a536443-95ba-4112-9369-b2df0407d010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:20.100920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a536443-95ba-4112-9369-b2df0407d010 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:20.102992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/60] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:20.108663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:20.206932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8a95e-4471-404a-a8ba-10254bfbbccb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:20.236516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:20.242669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ba9be-dd82-4de2-8d3e-8533bade9212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:20.243282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ba9be-dd82-4de2-8d3e-8533bade9212 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:20.243569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ba9be-dd82-4de2-8d3e-8533bade9212 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:20.243884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ba9be-dd82-4de2-8d3e-8533bade9212 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:20.244534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ba9be-dd82-4de2-8d3e-8533bade9212 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:20.245991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ba9be-dd82-4de2-8d3e-8533bade9212 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:20.272791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 31/61] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:20 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:23.334755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7b9dc-5f20-41a4-a610-6f3ff8a7b9ad None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.335616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7b9dc-5f20-41a4-a610-6f3ff8a7b9ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.336055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7b9dc-5f20-41a4-a610-6f3ff8a7b9ad None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.337833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:23.343626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.344447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.345008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.346528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e930c2-ef55-4ff5-a153-4cec275f203b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.347048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e930c2-ef55-4ff5-a153-4cec275f203b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.347524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e930c2-ef55-4ff5-a153-4cec275f203b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.349060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/63] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:23.354180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.354746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.355249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.369332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:23.376317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:23.384045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:23.391073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:23.495215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 32/64] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:23.517257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/65] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:23.518930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:23.540096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:23.665355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277dc71-a17a-40dc-a15f-430784f7a52f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:23.695791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:23.697380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b90074-bac6-4bd9-ac10-000909cfd1d9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:23.710908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8eae-aeb4-458e-8c73-4b233f5dbfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:23.712001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8eae-aeb4-458e-8c73-4b233f5dbfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.712487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8eae-aeb4-458e-8c73-4b233f5dbfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.713057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8eae-aeb4-458e-8c73-4b233f5dbfa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.720191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:23.729738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8eae-aeb4-458e-8c73-4b233f5dbfa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-452f8eae-aeb4-458e-8c73-4b233f5dbfa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:23.737212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:23.738052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:23.738869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:23.739981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea293d4-702f-4deb-99f7-330aa72b691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:23.740621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea293d4-702f-4deb-99f7-330aa72b691c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.740902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea293d4-702f-4deb-99f7-330aa72b691c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.741175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea293d4-702f-4deb-99f7-330aa72b691c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.742000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/66] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:23.749046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aad54-df42-4f97-9aad-2e11c4cf3025 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.749616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aad54-df42-4f97-9aad-2e11c4cf3025 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.750093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aad54-df42-4f97-9aad-2e11c4cf3025 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.752439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea293d4-702f-4deb-99f7-330aa72b691c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:23.752653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/67] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:23.753640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aad54-df42-4f97-9aad-2e11c4cf3025 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-dea293d4-702f-4deb-99f7-330aa72b691c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:23.758790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:23.759698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:23.761060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 33/68] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:23.761702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.761952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.762177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.783969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:23.785228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:23.877746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 34/69] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:23.878623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:23.898622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aad54-df42-4f97-9aad-2e11c4cf3025 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:23.938986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:23.940052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:23.940528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:23.940992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:23.941849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:23.942489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:23.944045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:23.953776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/70] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:23.954429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.003852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74446623-97bc-4487-85bf-3d15a577d495 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.004077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.012088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.048874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.049492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.050185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.050185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.050696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:24.050999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.052422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f868-2f1c-430f-8805-553fa688d610 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.066751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd69639-a80d-4a6a-bf8c-ebdeec76ea20 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.066919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 35/71] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:24.066919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7ff7-d515-4f6e-aaa1-e0f8703bb4ef None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.066919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7ff7-d515-4f6e-aaa1-e0f8703bb4ef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.066919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7ff7-d515-4f6e-aaa1-e0f8703bb4ef None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.066919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 36/72] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:24.066919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7ff7-d515-4f6e-aaa1-e0f8703bb4ef None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.111584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.112263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.112546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.112820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.123289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:24.128330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:24.129206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:24.129963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/73] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:24.130573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.173738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7ff7-d515-4f6e-aaa1-e0f8703bb4ef None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.208072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30b7c3-1499-486d-8f5f-8d92176d468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.208639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30b7c3-1499-486d-8f5f-8d92176d468a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.208928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30b7c3-1499-486d-8f5f-8d92176d468a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.209267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30b7c3-1499-486d-8f5f-8d92176d468a None admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.210417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30b7c3-1499-486d-8f5f-8d92176d468a None admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.211906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30b7c3-1499-486d-8f5f-8d92176d468a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.213480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 37/74] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:24.214585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b434862-8303-4944-b1e4-715938bf777f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.214875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b434862-8303-4944-b1e4-715938bf777f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.215154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b434862-8303-4944-b1e4-715938bf777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.232172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b434862-8303-4944-b1e4-715938bf777f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:24.233804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d24d72-7f60-4ee9-916e-21b3e972d6b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.233914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b434862-8303-4944-b1e4-715938bf777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:24.236135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.262963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.269304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d3c17-5716-4004-8fb3-d4aa52951a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.269942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d3c17-5716-4004-8fb3-d4aa52951a1d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.270226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d3c17-5716-4004-8fb3-d4aa52951a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.270504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d3c17-5716-4004-8fb3-d4aa52951a1d None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.285824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d3c17-5716-4004-8fb3-d4aa52951a1d None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.287301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d3c17-5716-4004-8fb3-d4aa52951a1d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.288970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/75] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:24.290209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ffd79-dc4c-41d2-9d6d-26b16aeb082e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.290497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ffd79-dc4c-41d2-9d6d-26b16aeb082e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.290778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ffd79-dc4c-41d2-9d6d-26b16aeb082e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.292017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/76] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:24.295744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ffd79-dc4c-41d2-9d6d-26b16aeb082e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.307320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 38/77] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:24.316409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b434862-8303-4944-b1e4-715938bf777f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.405240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ffd79-dc4c-41d2-9d6d-26b16aeb082e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.406967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b434862-8303-4944-b1e4-715938bf777f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.440161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.440769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.441054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.441333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.441813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf6b28-87a0-48a7-94f0-936ea8a05fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.441996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:24.442299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.442422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6b28-87a0-48a7-94f0-936ea8a05fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.442709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6b28-87a0-48a7-94f0-936ea8a05fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.442988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6b28-87a0-48a7-94f0-936ea8a05fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.443707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.452052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/78] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:24.452847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.477092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.477401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.512422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf6b28-87a0-48a7-94f0-936ea8a05fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.514482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf6b28-87a0-48a7-94f0-936ea8a05fe2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.554268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205fa1-7bdf-4046-ba83-24939f5e2600 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.588105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.588711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.588990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.589273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.611349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 39/79] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:24.612617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.612912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.613263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.630992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:24.632207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:24.655215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.656715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.681477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.691812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.699382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 40/80] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:24.709673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.761879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/81] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:24.762646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.809874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a885d13-7f2b-48d4-8840-8d456454fc9e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.844024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.844662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.844946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.845218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.856901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:24.860839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:24.861854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:24.862619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 41/82] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:24.863235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bcb65-26b1-44b3-9bf8-28f3c966dd08 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:24.863301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.886348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.898224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:24.898863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:24.899145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:24.899425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:24.900584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:24.902018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:24.903622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/83] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:24.905037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:24.926344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:24.970146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1181d-926e-487c-b7a8-258d98c77585 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.004338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548146ef-b14f-40cc-a39e-88a01a6e0ddc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.004539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.005096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.005379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.005660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.006819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.008314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.009932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 42/84] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.010696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.042029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.042596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.042882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.043164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.044264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.046701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.048324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/85] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.049083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.110288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cc8ed-60c1-4cc5-81a0-dc79ad90268e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.110823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.134318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.145533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.145902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.146190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.146468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.147620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.149088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.150912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 43/86] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.151018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618a5a2-472c-4f78-863a-a725bcd91aca None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.151486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.189953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.190562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.190844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.191130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.191818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:25.192126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.193514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.201302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/87] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.202084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.252463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a25647-f624-476b-ad5c-23f9b246c608 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.285828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.286466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.286754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.287031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.302213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20ca03-bce5-468c-84ee-cb15844129de None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.323950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.336233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.336836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.337116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.337387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.338108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=b26927358e664fe3981d02119bbe804c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:25.338411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.339861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.344469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.347733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/88] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/role_assignments?user.id=b26927358e664fe3981d02119bbe804c&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.360128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.369092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.370642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.474014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da66604-9bdf-4064-b4cd-b4743ce882d2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.486586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 44/89] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:25.486586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.522050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.522673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.522962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.523234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.523924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.525359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.553308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/90] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:25 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:25.553910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.602393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.604085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.616705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dfef5-8984-4eb8-9cff-66c795446487 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.666041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.666714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.667530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.667856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.668770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=e5ba3c8cd91e4ef6b8737d677d96472d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:25.669226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.671678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.681158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 45/91] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/role_assignments?user.id=e5ba3c8cd91e4ef6b8737d677d96472d&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.683346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff6ab4-ef13-4161-ae97-a3501523af48 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.686587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.726853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.727521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.727866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.728157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.729406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.731280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.734152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/92] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.740855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.841346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3aa9-8eef-488e-b69a-dfd3478e2c25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.881309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ff6b8-41b7-40e3-a6b6-d07303cbf320 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:25.885981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.886382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.886667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.886948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:25.888257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:25.891289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:25.893058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 46/93] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:25.897534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.899133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:25.927361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:25.928000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:25.929410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:25.929837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:25.930245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:26.022821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:26.024751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:26.036113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76e81-f4a2-4c99-a40a-039fa86fe457 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:26.083387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:26.084264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:26.084690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:26.085130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:26.086078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] RBAC: Adding query filter params (role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=e5ba3c8cd91e4ef6b8737d677d96472d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:26.087059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:26.088948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:26.099686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 47/94] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/role_assignments?user.id=e5ba3c8cd91e4ef6b8737d677d96472d&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:26.152070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/95] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:49:26.158124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:26.264557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:26.281025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a174c5-7b0d-4ebe-afdf-b2ff17e20a12 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:26.318490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249a00e-1777-4e3d-8b27-235a9a33d849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:26.319138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249a00e-1777-4e3d-8b27-235a9a33d849 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:26.319429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249a00e-1777-4e3d-8b27-235a9a33d849 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:26.319735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249a00e-1777-4e3d-8b27-235a9a33d849 None admin] PATH_INFO: `/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:26.321006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249a00e-1777-4e3d-8b27-235a9a33d849 None admin] RBAC: Authorizing `identity:get_project(project_id=5540a5dc579b4fea8615d90e0bb2aea0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:26.322990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249a00e-1777-4e3d-8b27-235a9a33d849 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:26.327717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 48/96] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:26 2022] GET /identity/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:26.331710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:26.344893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:26.447907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b689ff8-4a3c-4634-b57c-c787bf881e78 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:26.491809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:26.492524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:26.492779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:26.493061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:26.493806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=5540a5dc579b4fea8615d90e0bb2aea0, user.id=e5ba3c8cd91e4ef6b8737d677d96472d) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:26.494117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:26.495580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd9048-f1fd-4fdd-961f-7fb662523636 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:26.502222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/97] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:26 2022] GET /identity/v3/role_assignments?user.id=e5ba3c8cd91e4ef6b8737d677d96472d&scope.project.id=5540a5dc579b4fea8615d90e0bb2aea0&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:27.953776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83cde0-9c34-46c6-9ab9-0ef0e3c20a17 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:27.954729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83cde0-9c34-46c6-9ab9-0ef0e3c20a17 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:27.955235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83cde0-9c34-46c6-9ab9-0ef0e3c20a17 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:27.957135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 49/98] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:27.962066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:27.962877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:27.963389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:27.988491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:27.990077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:28.081017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/99] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:28.115023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83cde0-9c34-46c6-9ab9-0ef0e3c20a17 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:28.255736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83cde0-9c34-46c6-9ab9-0ef0e3c20a17 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:28.299267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fdd5-3392-4eb8-b6d9-af389c525301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:28.299962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fdd5-3392-4eb8-b6d9-af389c525301 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.300249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fdd5-3392-4eb8-b6d9-af389c525301 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.300529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fdd5-3392-4eb8-b6d9-af389c525301 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.311868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2fdd5-3392-4eb8-b6d9-af389c525301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6db2fdd5-3392-4eb8-b6d9-af389c525301 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:28.319270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:28.320138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:28.321762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 50/100] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:28.326430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:28.333440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:28.352639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca19d0-9d1f-438a-8ea2-63cc898ceb1b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.352945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca19d0-9d1f-438a-8ea2-63cc898ceb1b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.353206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca19d0-9d1f-438a-8ea2-63cc898ceb1b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.354948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 51/101] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:28.364479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.364751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.365085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.387066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:28.388308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:28.403923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:28.465572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 52/102] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:28.491514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:28.554465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ddfe2-9351-410b-a23a-a1c007aa1618 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:28.605185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53fe6e-179a-415f-85e2-9c210865828d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:28.619105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:28.619683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.619974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.620550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.621240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:28.621657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:28.623818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:28.641707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/103] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:28.643893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:28.644601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.644881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.645160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.648045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:28.656700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:28.661618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:28.662466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:28.663208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 53/104] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:28.670830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:28.695988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:28.770450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:28.820247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fce19-c13a-4262-94bb-11e471eec135 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:28.832763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046598d-2e28-4060-8d3a-b72deb7297d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:28.882154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:28.883100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.883578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.884071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.884991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:28.885491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:28.887128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:28.902904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 54/105] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:28.909545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:28.909852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:28.910272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:28.910565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:28.915260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:28.915260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:28.915260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:28.915260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/106] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:28.922045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.012303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.045413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db19dd-c19b-44aa-ad87-a17be094b6b3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.085134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3bc7-172b-428a-8e83-5772363f882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.086261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3bc7-172b-428a-8e83-5772363f882c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.086768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3bc7-172b-428a-8e83-5772363f882c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.087239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3bc7-172b-428a-8e83-5772363f882c None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.089269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3bc7-172b-428a-8e83-5772363f882c None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:29.092881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3bc7-172b-428a-8e83-5772363f882c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:29.095460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 55/107] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:29.101818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854c5c9-ff03-491d-8c59-5bdadbd3e466 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.104276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35314aab-8fc7-4c80-bbb8-b987c87b173c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.106621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854c5c9-ff03-491d-8c59-5bdadbd3e466 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.108328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854c5c9-ff03-491d-8c59-5bdadbd3e466 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.109919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 56/108] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:29.111865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854c5c9-ff03-491d-8c59-5bdadbd3e466 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.159062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f90b7-ace7-403b-b94c-d323bf3dad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.160123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f90b7-ace7-403b-b94c-d323bf3dad0b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.161016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f90b7-ace7-403b-b94c-d323bf3dad0b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.161518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f90b7-ace7-403b-b94c-d323bf3dad0b None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.162880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f90b7-ace7-403b-b94c-d323bf3dad0b None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:29.165191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f90b7-ace7-403b-b94c-d323bf3dad0b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:29.167411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/109] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:28 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:29.169346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.169941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.170696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.178419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.194350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:29.195875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:29.288040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/110] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:29.288746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.294772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854c5c9-ff03-491d-8c59-5bdadbd3e466 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.332917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec0c8d-8f84-45ab-aac8-10a5cc17d5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.333891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec0c8d-8f84-45ab-aac8-10a5cc17d5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.334443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec0c8d-8f84-45ab-aac8-10a5cc17d5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.334927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec0c8d-8f84-45ab-aac8-10a5cc17d5b5 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.336295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec0c8d-8f84-45ab-aac8-10a5cc17d5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:29.338165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec0c8d-8f84-45ab-aac8-10a5cc17d5b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:29.340406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 57/111] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:29 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:29.347067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e829484-8227-4cb0-ba45-e6f23cce019d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.347728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e829484-8227-4cb0-ba45-e6f23cce019d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.348289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e829484-8227-4cb0-ba45-e6f23cce019d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.349923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 58/112] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:29.350850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e829484-8227-4cb0-ba45-e6f23cce019d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.360106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.411387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d471c6-1156-4e82-b16c-c7f68897dec4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.449906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.457533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.458799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.459432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.459914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/b26927358e664fe3981d02119bbe804c/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.461864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec2591650a2e438db15b3fccbca1ca57, user_id=b26927358e664fe3981d02119bbe804c, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:29.463320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:29.476208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/113] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:29 2022] PUT /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/b26927358e664fe3981d02119bbe804c/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:29.477367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.490533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e829484-8227-4cb0-ba45-e6f23cce019d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.532875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247000b-c8d7-47f9-80a0-610915edf688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.533716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247000b-c8d7-47f9-80a0-610915edf688 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.534068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247000b-c8d7-47f9-80a0-610915edf688 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.534505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247000b-c8d7-47f9-80a0-610915edf688 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.548554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247000b-c8d7-47f9-80a0-610915edf688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4247000b-c8d7-47f9-80a0-610915edf688 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:29.553861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:29.554742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 59/114] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:29.560995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1549-f61a-4ec8-a352-895ffcee8851 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.561842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1549-f61a-4ec8-a352-895ffcee8851 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.561842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1549-f61a-4ec8-a352-895ffcee8851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.581352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-509a1549-f61a-4ec8-a352-895ffcee8851 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:29.583095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-509a1549-f61a-4ec8-a352-895ffcee8851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:29.642479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6dfa2-d7b7-4a75-ba6a-44b713722af0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.656411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.666558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 60/115] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:29.668063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238081c-84df-4328-a9a6-a689e5113d17 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.668430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238081c-84df-4328-a9a6-a689e5113d17 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.668806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238081c-84df-4328-a9a6-a689e5113d17 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.670703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 61/116] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:29.671552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238081c-84df-4328-a9a6-a689e5113d17 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.696806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fe7d-039a-4339-bb68-f6550e263b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.697724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fe7d-039a-4339-bb68-f6550e263b1a None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.698115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fe7d-039a-4339-bb68-f6550e263b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.698399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fe7d-039a-4339-bb68-f6550e263b1a None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/e5ba3c8cd91e4ef6b8737d677d96472d/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.701831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92fe7d-039a-4339-bb68-f6550e263b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=ec2591650a2e438db15b3fccbca1ca57, user_id=e5ba3c8cd91e4ef6b8737d677d96472d, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:29.703512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92fe7d-039a-4339-bb68-f6550e263b1a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:29.719175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/117] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:29 2022] PUT /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/e5ba3c8cd91e4ef6b8737d677d96472d/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:29.720422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.720706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.720983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.749770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:29.752393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:29.813969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.817115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238081c-84df-4328-a9a6-a689e5113d17 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:29.843407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/118] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:29.844427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.864238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:29.864870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.865145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.865417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.866107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:29.866411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:29.868098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aedbf-7177-4b59-bfdd-b28efc4e95c1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:29.884128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 62/119] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:29.886570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92054a57-527c-45b6-b580-913bfa8216a3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:29.887525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92054a57-527c-45b6-b580-913bfa8216a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:29.888275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92054a57-527c-45b6-b580-913bfa8216a3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:29.891051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 63/120] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:29.892242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92054a57-527c-45b6-b580-913bfa8216a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:29.961603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:29.988614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a601cca-7917-46d5-9214-ab2f006b2951 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.025531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92054a57-527c-45b6-b580-913bfa8216a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.030620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.031390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.031720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.032001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.044030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:30.050096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:30.050962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:30.051793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/121] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:30.052411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.078269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cf67b-86dd-4d71-98e9-27ae5e2fed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.079495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cf67b-86dd-4d71-98e9-27ae5e2fed0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.080454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cf67b-86dd-4d71-98e9-27ae5e2fed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.080937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cf67b-86dd-4d71-98e9-27ae5e2fed0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.082352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cf67b-86dd-4d71-98e9-27ae5e2fed0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.084914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cf67b-86dd-4d71-98e9-27ae5e2fed0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.088081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 64/122] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:30.090397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.090917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.091639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.115771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:30.118041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:30.124221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.171836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef6fad-6fed-443e-9fad-d8a50f833383 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.209821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 65/123] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:30.211425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.214547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.215140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.215422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.215752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] PATH_INFO: `/v3/users/f5542ae0e7a443cbac829620e1e6a3d8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.240121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] RBAC: Authorizing `identity:get_user(user_id=f5542ae0e7a443cbac829620e1e6a3d8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.241619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.245765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/124] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/users/f5542ae0e7a443cbac829620e1e6a3d8 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:30.246774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.281905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.329029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84e14e-483c-46ad-9c53-2d185b322f0e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.365104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.365726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.366010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.366332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.367015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:30.367327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.368730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.372817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.378357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b1c6-d63d-4bc1-b79b-9eba24d2d546 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.382148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 66/125] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:30.383073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.417385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.418448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.419000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.419475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.420907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.422581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.453681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/126] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:49:30 2022] POST /identity/v3/projects => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:30.455273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.490578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e371fb8-b6cf-4076-aab6-e91141cc556b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.514565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.540865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.542167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.543050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.543693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.559934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:30.567713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:30.568555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:30.569305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:30.577466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 67/127] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:30.577466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.600883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275c0b-01d3-40f0-a86e-2acfa06985bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.644034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.645042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.645526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.645998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.678396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.744087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.745737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.768003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022901-b659-4640-ae80-8f9f67154f86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.803852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.804788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.805207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.805665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.806939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.808573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.811363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 68/128] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:30.813217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.864771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.874309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/129] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:30.875432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:30.925242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637d92f-30b8-4220-8a36-e7464c6e7e00 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:30.934480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:30.966687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:30.967647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:30.968067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:30.968556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:30.969516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:30.969947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:30.971631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:30.980494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 69/130] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:30.981540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:31.025861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74e80f-9b17-4bc3-ae1d-8c1bdbe1306a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.087514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae267497-79f2-4022-8229-6d82c9ec6f35 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.098172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:31.120864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.121701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.121809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.122070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.127542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.132474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.133303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.134963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.136147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/131] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:31.136633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.137107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.138688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:31.139517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.141495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.145716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 70/132] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:30 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:31.146773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:31.308055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6be16-1564-496c-8c71-389e02ee437c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.337926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf360c0a-c4e0-4863-be90-05e0980a08d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.345132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.346072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.346554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.347108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/e5ba3c8cd91e4ef6b8737d677d96472d/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.349554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] RBAC: Authorizing `identity:create_grant(project_id=ec2591650a2e438db15b3fccbca1ca57, user_id=e5ba3c8cd91e4ef6b8737d677d96472d, role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.351206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.371275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 71/133] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:31 2022] PUT /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/e5ba3c8cd91e4ef6b8737d677d96472d/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:31.372658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:31.380677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:31.410828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.411883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.412390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.412864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.415042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.419937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.423276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/134] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:31 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:31.428519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:31.454535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:31.541307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082149a-1323-41de-a4eb-57e4f08bd01c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.584891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3eb358-fdcc-4fbc-a323-883cf3155f9a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.595433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.596105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.596531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.596813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.600947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] RBAC: Adding query filter params (role.id=7d1dedab3b264ed8816f26bb28b74647, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=f5542ae0e7a443cbac829620e1e6a3d8) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:31.601269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.602790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.619264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 72/135] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:31 2022] GET /identity/v3/role_assignments?user.id=f5542ae0e7a443cbac829620e1e6a3d8&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=7d1dedab3b264ed8816f26bb28b74647 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:31.632472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c796c-113a-4b27-98e8-f1dfeb20316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.633245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c796c-113a-4b27-98e8-f1dfeb20316e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.633530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c796c-113a-4b27-98e8-f1dfeb20316e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.633808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c796c-113a-4b27-98e8-f1dfeb20316e None admin] PATH_INFO: `/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.635076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c796c-113a-4b27-98e8-f1dfeb20316e None admin] RBAC: Authorizing `identity:get_project(project_id=5540a5dc579b4fea8615d90e0bb2aea0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.636795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c796c-113a-4b27-98e8-f1dfeb20316e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.639299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/136] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:31 2022] GET /identity/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:31.645731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:31.768936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:31.773087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d47173-0f14-4cee-9b50-113eddeeac24 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:31.816877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6c3cb-1e7f-44a0-a26e-19028cb40688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:31.817565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6c3cb-1e7f-44a0-a26e-19028cb40688 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:31.817848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6c3cb-1e7f-44a0-a26e-19028cb40688 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:31.818124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6c3cb-1e7f-44a0-a26e-19028cb40688 None admin] PATH_INFO: `/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0/users/e5ba3c8cd91e4ef6b8737d677d96472d/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:31.820197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6c3cb-1e7f-44a0-a26e-19028cb40688 None admin] RBAC: Authorizing `identity:create_grant(project_id=5540a5dc579b4fea8615d90e0bb2aea0, user_id=e5ba3c8cd91e4ef6b8737d677d96472d, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:31.821689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6c3cb-1e7f-44a0-a26e-19028cb40688 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:31.838585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 73/137] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:31 2022] PUT /identity/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0/users/e5ba3c8cd91e4ef6b8737d677d96472d/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:32.012480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe44cd0-689a-47d6-ba1e-cac4732aec16 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.012822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe44cd0-689a-47d6-ba1e-cac4732aec16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.013033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe44cd0-689a-47d6-ba1e-cac4732aec16 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.015659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/138] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:32.022353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0621700-a897-4074-aaf4-521ffd99aa92 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.022646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0621700-a897-4074-aaf4-521ffd99aa92 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.022922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0621700-a897-4074-aaf4-521ffd99aa92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.047648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a0621700-a897-4074-aaf4-521ffd99aa92 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:32.048856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a0621700-a897-4074-aaf4-521ffd99aa92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:32.141146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 74/139] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:32.157840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f2ff-422f-471d-9dc9-03029a6a19f3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.158330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f2ff-422f-471d-9dc9-03029a6a19f3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.158410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f2ff-422f-471d-9dc9-03029a6a19f3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.160038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/140] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:32.165106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.165391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.165664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.182829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f2ff-422f-471d-9dc9-03029a6a19f3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:32.188489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:32.189683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:32.306307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 75/141] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:32.312935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:32.358984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:32.396427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:32.445250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f2ff-422f-471d-9dc9-03029a6a19f3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:32.515841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:32.516593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.516853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.517147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.535468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:32.548531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:32.549445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:32.550648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/142] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:32.562351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:32.563577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a23b1-f230-4d8f-918f-549edfda34da None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:32.619155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:32.619952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.620240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.620513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.639860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:32.644771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:32.645611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:32.646888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 76/143] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:32.652268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:32.792002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:32.829797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b869-e858-4437-baf6-5f6951872ffe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:32.911365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e12b5b-c6c4-45be-8e6f-3672baff9ee6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:32.947994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:32.976342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:32.976985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.977270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.977547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.978233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:32.978540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:32.979977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:32.981479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:32.982652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:32.982939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:32.983219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:32.983935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:32.984248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:32.985630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:33.003443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 77/144] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:33.005448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/145] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:33.009130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:33.011496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:33.222098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef82ff9-9d70-4af7-9c4f-6a61c1d3e465 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:33.267709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:33.268383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:33.268666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:33.268942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:33.274017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:33.275495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:33.277413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/146] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:33 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:33.283800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:33.317846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82610653-65fb-4b65-83a2-f9519923fc20 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:33.391952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:33.418277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:33.419383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:33.420490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:33.421001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] PATH_INFO: `/v3/users/b26927358e664fe3981d02119bbe804c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:33.428351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] RBAC: Authorizing `identity:get_user(user_id=b26927358e664fe3981d02119bbe804c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:33.429846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:33.431489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 78/147] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:33 2022] GET /identity/v3/users/b26927358e664fe3981d02119bbe804c => generated 247 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:33.437268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:33.460578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:33.505934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae13d-668a-43c1-95f7-634a1fa03e36 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:33.568278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc7cf-eec0-46b3-a669-ba29727710de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:33.568907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc7cf-eec0-46b3-a669-ba29727710de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:33.569201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc7cf-eec0-46b3-a669-ba29727710de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:33.569485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc7cf-eec0-46b3-a669-ba29727710de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:33.622466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227a0cf-0bdc-4c0b-9f00-13418a7ab751 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:33.662201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbbe2f-6d7d-4f73-9f1c-d83146664293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:33.662853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbbe2f-6d7d-4f73-9f1c-d83146664293 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:33.663139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbbe2f-6d7d-4f73-9f1c-d83146664293 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:33.663417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbbe2f-6d7d-4f73-9f1c-d83146664293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:33.709566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc7cf-eec0-46b3-a669-ba29727710de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:33.711273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc7cf-eec0-46b3-a669-ba29727710de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:33.752169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbbe2f-6d7d-4f73-9f1c-d83146664293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:33.753858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbbe2f-6d7d-4f73-9f1c-d83146664293 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:33.765755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:33.807388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:33.895896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/148] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:49:33.897100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949c35d-45bb-4780-936a-d81ab90202e5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:33.897401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949c35d-45bb-4780-936a-d81ab90202e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:33.897686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949c35d-45bb-4780-936a-d81ab90202e5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:33.898938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/149] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:33.903739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949c35d-45bb-4780-936a-d81ab90202e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:33.939305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 79/150] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:49:33.941018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:33.941540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:33.942075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:33.979086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:33.980343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:34.067868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949c35d-45bb-4780-936a-d81ab90202e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:34.101274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 80/151] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:34.102138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.142105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:34.142815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.143100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.143387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.144597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:34.146063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:34.147791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/152] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:33 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:34.151885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.158102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:34.183970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:34.226145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c950-5bf0-4fc1-b5e2-1ca49bff55f9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:34.301149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:34.301797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.302073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.302346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.309502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:34.311901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:34.313610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 81/153] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:34 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:34.314388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.369254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e5bf-ee30-45bc-b1fc-9d3a9fb4f520 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:34.423021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:34.424133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.424430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.424890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.426001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:34.427451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:34.429392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/154] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:34.430184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.441014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7269c-bac8-468d-8ebe-c305c883facd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:34.450485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:34.480996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:34.481600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.481876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.482695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.483827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=e5ba3c8cd91e4ef6b8737d677d96472d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:34.484146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:34.485579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c8934-9075-4a6e-a349-3d684aad8a68 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:34.495635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 82/155] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:34 2022] GET /identity/v3/role_assignments?user.id=e5ba3c8cd91e4ef6b8737d677d96472d&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:34.495635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57dc7-56da-4cd5-aa53-38fdc542af6a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.495635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57dc7-56da-4cd5-aa53-38fdc542af6a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.495635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57dc7-56da-4cd5-aa53-38fdc542af6a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.495635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 83/156] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:34.495635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8caec-1479-4580-bac6-435c98201b6e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.496082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8caec-1479-4580-bac6-435c98201b6e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.496082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8caec-1479-4580-bac6-435c98201b6e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.498631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 84/157] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:34.498631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8caec-1479-4580-bac6-435c98201b6e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.538328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:34.562074 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27870bbe-85b4-4ceb-93be-b3ed805f2a84 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:34.601141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:34.601907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.602308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.602679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.603531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=b26927358e664fe3981d02119bbe804c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:34.603975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:34.605532 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2c7b1-04af-4fcd-ab16-52cff17151ce None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:34.615248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/158] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:34 2022] GET /identity/v3/role_assignments?user.id=b26927358e664fe3981d02119bbe804c&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:34.616542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3168-90ef-4c84-bc00-eba7efdd82cf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.616930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3168-90ef-4c84-bc00-eba7efdd82cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.617305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3168-90ef-4c84-bc00-eba7efdd82cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.639328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-672d3168-90ef-4c84-bc00-eba7efdd82cf None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:34.640912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-672d3168-90ef-4c84-bc00-eba7efdd82cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:34.648243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8caec-1479-4580-bac6-435c98201b6e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:34.690653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238080a-2e15-4f7b-9e01-27ae09c14d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:34.691284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238080a-2e15-4f7b-9e01-27ae09c14d93 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.691562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238080a-2e15-4f7b-9e01-27ae09c14d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.691864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238080a-2e15-4f7b-9e01-27ae09c14d93 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.692530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238080a-2e15-4f7b-9e01-27ae09c14d93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:34.694028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238080a-2e15-4f7b-9e01-27ae09c14d93 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:34.723182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 85/159] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:34 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:34.724655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.725333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.725721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.730382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/160] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:34.731956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd456c-dd63-4df4-8008-dcb8c8c60e18 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.732279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd456c-dd63-4df4-8008-dcb8c8c60e18 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.732565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd456c-dd63-4df4-8008-dcb8c8c60e18 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.734150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/161] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:34.740032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:34.740434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:34.740721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:34.746679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:34.748390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:34.760698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:34.761940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:34.780456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:34.843976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 86/162] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:34.845209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.853103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/163] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:34.881973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:34.891747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:34.991915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a39d9-cba3-4aee-bd35-25c869ab5c35 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.015887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2a2a0-2b6d-4da8-8757-5da1305336a2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.038906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.039962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.039962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.040138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.055634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:35.059258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:35.060666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:35.061495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:35.062349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:35.062929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.062929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.062929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.063098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 87/164] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:35.063171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.082272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:35.087079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:35.088011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:35.089058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/165] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:35.089824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.090913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:35.183365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:35.205190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42b71-1d6d-4ae4-950a-e68ee3e92c69 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.230309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2cd1f-c1f5-4efc-9b22-8021b0aef4e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.256456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.257194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.257551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.257902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.273033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:35.278250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:35.278814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:35.280463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:35.281222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:35.281833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.281833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.281833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.281833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:35.281833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:35.282186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 88/166] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:35.283634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.283830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:35.294052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/167] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:35.294993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.397541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:35.428598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed491c1-9075-4cfe-90d3-8831f61abc96 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.429935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfde3f-4412-4293-98e7-23c268fb3910 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.463791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:35.473597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.474217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.474493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.475253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.476011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:35.476315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:35.477594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.478056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:35.478866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.479339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.479839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.480726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:35.481231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:35.482943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:35.489000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/168] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:35.489938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.493708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 89/169] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:35.494790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.648557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ae264-3a1e-4642-a3a4-18fb69751334 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.674592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b893f-a59b-4cdf-8347-67771a694097 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.686716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.687743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.688222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.688698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.690076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:35.691744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:35.693979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 90/170] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:35.695049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.705262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:35.733934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.735302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.738910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.739463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] PATH_INFO: `/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.740996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] RBAC: Authorizing `identity:get_user(user_id=e5ba3c8cd91e4ef6b8737d677d96472d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:35.742624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:35.746155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/171] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/users/e5ba3c8cd91e4ef6b8737d677d96472d => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:35.749443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:35.867294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:35.906606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026f38f-7241-4c0f-b9b1-338dfb040428 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.954763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04adf3-b2c4-4a3f-87a5-89be41fd48d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:35.976366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:35.977008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:35.977285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:35.977554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] PATH_INFO: `/v3/users/f5542ae0e7a443cbac829620e1e6a3d8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:35.979545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] RBAC: Authorizing `identity:get_user(user_id=f5542ae0e7a443cbac829620e1e6a3d8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:35.982511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:35.985259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 91/172] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/users/f5542ae0e7a443cbac829620e1e6a3d8 => generated 289 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:35.991490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:36.050655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:36.051641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:36.052115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:36.052604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:36.075002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:36.146896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:36.148710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:36.168480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48526232-3bf8-4e69-b5c2-3d031a0a1c3c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:36.215976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:36.262559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/173] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:36.263317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:36.324348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:36.324983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:36.325259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:36.325601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:36.425671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7f7f5-a5fc-4225-8a1f-ed4587d8bd8c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:36.429714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:36.431179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:36.515288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:36.529661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:36.530412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:36.530689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:36.530957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:36.536494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:36.537994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:36.539787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/174] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:36 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:36.540591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:36.547124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 92/175] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:36.548039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:36.620951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:36.663011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f98b76-e02c-432a-bb48-73f96187fe17 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:36.702683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c0460-3053-410f-9657-c979988f805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:36.703483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c0460-3053-410f-9657-c979988f805c None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:36.703794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c0460-3053-410f-9657-c979988f805c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:36.704070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c0460-3053-410f-9657-c979988f805c None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/f5542ae0e7a443cbac829620e1e6a3d8/roles/7d1dedab3b264ed8816f26bb28b74647` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:36.706215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c0460-3053-410f-9657-c979988f805c None admin] RBAC: Authorizing `identity:create_grant(project_id=ec2591650a2e438db15b3fccbca1ca57, user_id=f5542ae0e7a443cbac829620e1e6a3d8, role_id=7d1dedab3b264ed8816f26bb28b74647)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:36.708171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c0460-3053-410f-9657-c979988f805c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:36.724557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 93/176] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:36 2022] PUT /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57/users/f5542ae0e7a443cbac829620e1e6a3d8/roles/7d1dedab3b264ed8816f26bb28b74647 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:36.725416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c0460-3053-410f-9657-c979988f805c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:36.805965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:36.829061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d1caa-f1c4-4ace-b4e5-8d725504df1c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:36.924429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:36.925097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:36.925378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:36.925656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] PATH_INFO: `/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:36.926946 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] RBAC: Authorizing `identity:get_project(project_id=5540a5dc579b4fea8615d90e0bb2aea0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:36.930069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:36.930224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/177] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:36 2022] GET /identity/v3/projects/5540a5dc579b4fea8615d90e0bb2aea0 => generated 312 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:36.938010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:36.987963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c0460-3053-410f-9657-c979988f805c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.036636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.037284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.037560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.037833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.039554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:37.041399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:37.043524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 94/178] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:36 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:37.050793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:37.146271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:37.165761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11154f4d-7e2d-43af-9860-ba900ed12650 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.172126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93bd97-36a8-4d81-a5eb-03c8ff9c1752 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.174961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:37.214850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.216337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.217029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.217817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.218690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] RBAC: Adding query filter params (role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=e5ba3c8cd91e4ef6b8737d677d96472d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:37.219131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:37.220921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f66faa-4ee8-4f2e-a906-a73ce6f0acfa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:37.227904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 95/179] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:37 2022] GET /identity/v3/role_assignments?user.id=e5ba3c8cd91e4ef6b8737d677d96472d&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:37.258453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.259087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.259366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.259661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.260429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=5540a5dc579b4fea8615d90e0bb2aea0, user.id=e5ba3c8cd91e4ef6b8737d677d96472d) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:37.260737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:37.262222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f2aca-4e87-4d61-b0a0-9ec969df5bd6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:37.276846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/180] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:36 2022] GET /identity/v3/role_assignments?user.id=e5ba3c8cd91e4ef6b8737d677d96472d&scope.project.id=5540a5dc579b4fea8615d90e0bb2aea0&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 608 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:37.364574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8e6b1-455d-4c07-82c9-7086d69170e1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.365219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8e6b1-455d-4c07-82c9-7086d69170e1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.365694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8e6b1-455d-4c07-82c9-7086d69170e1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.367250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 96/181] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:37.370967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bd2c5-17ff-40bf-add2-089a40262725 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.371266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bd2c5-17ff-40bf-add2-089a40262725 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.371527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bd2c5-17ff-40bf-add2-089a40262725 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.373009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/182] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:37.377468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.378038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.378501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.378929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.379217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.379489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.402144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:37.404918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:37.416756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:37.417994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:37.494871 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 97/183] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:37.534951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:37.581309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/184] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:37.604486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:37.655941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:37.663240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61f90b-53f1-4cff-a404-83ed64cdc8bc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.709226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.710653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.711155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.711663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.726130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:37.727258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:37.733122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:37.734065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:37.734824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:37.737006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 98/185] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:37.744520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:37.754513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4aa7-0a09-4fbd-b3bf-d5872c3c5f2d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.799085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.799904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.800274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.800624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.812966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:37.818756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:37.819782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:37.820547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/186] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:37.825648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:37.875213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e671bf-b4a7-4032-a585-6116ececbb57 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.910700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.911238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.911524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.911849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.912522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:37.912829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:37.914231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd9ebb-af0a-4c5d-a5da-68cc8ac95403 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:37.927745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 99/187] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:37.927745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a521-a6df-456b-b5ac-23756aa8e029 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.927745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a521-a6df-456b-b5ac-23756aa8e029 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.927745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a521-a6df-456b-b5ac-23756aa8e029 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.927745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 100/188] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:37.929717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a521-a6df-456b-b5ac-23756aa8e029 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:37.940935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b965252-4e23-4d61-ad1f-fb253e51f32f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:37.949496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:37.977937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:37.978543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.978818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.979091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:37.979783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:37.980084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:37.981440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6be3-434f-4eda-98e1-452a6f455b17 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:37.990578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/189] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:37.996133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:37.996418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:37.996820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.014977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:38.016205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:38.019736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.037905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a521-a6df-456b-b5ac-23756aa8e029 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.073741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.074629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.074913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.075193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] PATH_INFO: `/v3/users/9243e16a86944f888cf505df2df68396` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.089096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] RBAC: Authorizing `identity:get_user(user_id=9243e16a86944f888cf505df2df68396)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.099616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/190] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:38.099772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.099895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.099988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 101/191] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:37 2022] GET /identity/v3/users/9243e16a86944f888cf505df2df68396 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:38.099988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.181293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.206653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0cf4-06cd-42c7-8fa0-5dea9085211e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.211186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f5e27-ac22-4cb4-97c4-d23702dfffef None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.245270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.245697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.245897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.246146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.246359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.246455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.246610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.246938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] PATH_INFO: `/v3/users/9243e16a86944f888cf505df2df68396` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.248043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] RBAC: Authorizing `identity:get_user(user_id=9243e16a86944f888cf505df2df68396)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.249485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.251249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/192] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/users/9243e16a86944f888cf505df2df68396 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:38.252270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.264238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.322937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.324467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.374481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002474e1-3de6-4d40-b041-96e547b26551 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.403749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.413847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.414486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.414769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.415051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.416194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.417634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.419212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/193] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:38.420822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.434785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 102/194] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:49:38.436158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.504021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.527967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979f4f-49e7-4830-ac1e-851199c83803 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.547711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1729-748a-438e-911f-aae2eebe98df None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.562733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.563343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.563668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.563947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.584034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.584642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.584918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.585246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.585882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.587743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.617813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 103/195] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:38 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:38.617813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.652373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.654209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.656443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.744692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.767899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a8a2d-56d2-421d-9a72-47a536dd4a8f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.776260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/196] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:38.782285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.820682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.822130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.822406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.822678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.823866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.825309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.827801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 104/197] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:38.834191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:38.906847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1216a01-fcde-4e23-a45c-ba25061aa68c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.957479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:38.958634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:38.959402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:38.959967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:38.961702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:38.963089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:38.964187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:38.966854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/198] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:38.969909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2e9d4-94cf-449b-8223-aae268165b0d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:38.974216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:39.010445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:39.010991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:39.011265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:39.011540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:39.012315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=9243e16a86944f888cf505df2df68396) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:39.012627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:39.014710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fac8a-a5e7-44a4-a675-1a848cb25031 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:39.020757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 105/199] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/role_assignments?user.id=9243e16a86944f888cf505df2df68396&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:39.027749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:39.096232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03de45-1aed-4cbb-bbd3-f144a1bca742 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:39.137560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:39.138629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:39.139104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:39.139614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:39.140563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] RBAC: Adding query filter params (role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=9243e16a86944f888cf505df2df68396) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:39.141070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:39.142833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528010c-0c42-46a2-8f5e-64c4d234e909 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:39.149607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/200] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:38 2022] GET /identity/v3/role_assignments?user.id=9243e16a86944f888cf505df2df68396&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:39.286111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f3c60-d9e3-4889-8895-76a458e05d39 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:39.286452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f3c60-d9e3-4889-8895-76a458e05d39 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:39.286621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f3c60-d9e3-4889-8895-76a458e05d39 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:39.288294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 106/201] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:39.294059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:39.294414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:39.294670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:39.317284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:39.318495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:39.401483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/202] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:39.436693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f3c60-d9e3-4889-8895-76a458e05d39 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:39.585512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f3c60-d9e3-4889-8895-76a458e05d39 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:39.631788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:39.632787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:39.633269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:39.633843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:39.651398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:39.656649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:39.657521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:39.658312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:39.661319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 107/203] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:39.668738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:39.734618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:39.816443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b04efd-33d4-4dde-956c-658adb87daad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:39.864926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:39.865557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:39.865840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:39.866114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:39.866789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:39.867092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:39.868491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:39.883180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/204] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:39.889580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:39.943514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:40.048496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb51a-5422-442b-8b7b-102e37b7249d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:40.098576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:40.099214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:40.099496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:40.099799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] PATH_INFO: `/v3/users/f5542ae0e7a443cbac829620e1e6a3d8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:40.104278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] RBAC: Authorizing `identity:get_user(user_id=f5542ae0e7a443cbac829620e1e6a3d8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:40.105945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:40.107908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 108/205] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:39 2022] GET /identity/v3/users/f5542ae0e7a443cbac829620e1e6a3d8 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:40.113509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:40.231078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:40.234743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2611b1-0dff-4f89-8c5c-64b3331b56d9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:40.271265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:40.271907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:40.272187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:40.272462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:40.359256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:40.360740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:40.462856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:40.466387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/206] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:40.471852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:40.590841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:40.602280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137d62-7744-414e-a522-ab05ba57a9fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:40.651671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11b59d-acfe-4d9a-a53d-172151867cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:40.652319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b59d-acfe-4d9a-a53d-172151867cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:40.652596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b59d-acfe-4d9a-a53d-172151867cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:40.652868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b59d-acfe-4d9a-a53d-172151867cd8 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:40.656274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11b59d-acfe-4d9a-a53d-172151867cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:40.657788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11b59d-acfe-4d9a-a53d-172151867cd8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:40.659466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 109/207] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:40 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:40.665423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:40.807840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f05b7-6abf-44a8-8a9c-3a5d27a67059 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:40.866270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:40.866897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:40.867176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:40.867447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:40.868213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] RBAC: Adding query filter params (role.id=7d1dedab3b264ed8816f26bb28b74647, scope.project.id=ec2591650a2e438db15b3fccbca1ca57, user.id=f5542ae0e7a443cbac829620e1e6a3d8) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:40.868520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:40.869969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6d42e-c505-495e-ba6e-4bd97eb5f1ce None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:40.880832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/208] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:40 2022] GET /identity/v3/role_assignments?user.id=f5542ae0e7a443cbac829620e1e6a3d8&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=7d1dedab3b264ed8816f26bb28b74647 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:41.303402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427a79-2744-43d3-a442-708ec078eea1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.303402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427a79-2744-43d3-a442-708ec078eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.303402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427a79-2744-43d3-a442-708ec078eea1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.304186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 110/209] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:41.309828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8dfa-de29-4e97-bf62-51bc24cadffb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.310906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8dfa-de29-4e97-bf62-51bc24cadffb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.311399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8dfa-de29-4e97-bf62-51bc24cadffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.330753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-710a8dfa-de29-4e97-bf62-51bc24cadffb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:41.332322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-710a8dfa-de29-4e97-bf62-51bc24cadffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:41.394381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:41.403382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/210] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:41.409983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b538-79fd-4936-92b0-be96d262f6c2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.410562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b538-79fd-4936-92b0-be96d262f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.411044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b538-79fd-4936-92b0-be96d262f6c2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.412626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 111/211] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:41.419275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.419863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.420340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.435533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b538-79fd-4936-92b0-be96d262f6c2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:41.441820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:41.443020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:41.524340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/212] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:41.551289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:41.568823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b538-79fd-4936-92b0-be96d262f6c2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:41.614908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:41.615533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.615862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.616125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.617533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:41.619679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:41.621795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 112/213] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:41.627655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:41.664802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:41.667045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44976622-6e14-42ff-b864-20609ac55c15 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:41.694921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:41.708625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27423cc-18fe-4c9e-8faa-42fa7cb99cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:41.709464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27423cc-18fe-4c9e-8faa-42fa7cb99cbd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.709909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27423cc-18fe-4c9e-8faa-42fa7cb99cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.710327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27423cc-18fe-4c9e-8faa-42fa7cb99cbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.722027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27423cc-18fe-4c9e-8faa-42fa7cb99cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c27423cc-18fe-4c9e-8faa-42fa7cb99cbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:41.726968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:41.727994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:41.728884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:41.730791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/214] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:41.733344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787badc-432e-4376-b154-7c0ce95082d8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.733786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787badc-432e-4376-b154-7c0ce95082d8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.734224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787badc-432e-4376-b154-7c0ce95082d8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.735672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/215] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:41.737195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787badc-432e-4376-b154-7c0ce95082d8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:41.748983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e495e-ad01-4d84-8602-7b90b1a72fb2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:41.783122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b0844-7fbe-4db8-b04e-ddf66c496d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:41.783766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b0844-7fbe-4db8-b04e-ddf66c496d4a None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.784050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b0844-7fbe-4db8-b04e-ddf66c496d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.784332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b0844-7fbe-4db8-b04e-ddf66c496d4a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.784968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b0844-7fbe-4db8-b04e-ddf66c496d4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:41.786403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b0844-7fbe-4db8-b04e-ddf66c496d4a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:41.817087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 113/216] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:41 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:49:41.817087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f589a-c021-4b53-a17a-617709bfeae7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.817087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f589a-c021-4b53-a17a-617709bfeae7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.817087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f589a-c021-4b53-a17a-617709bfeae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.833103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ce1f589a-c021-4b53-a17a-617709bfeae7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:41.834231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ce1f589a-c021-4b53-a17a-617709bfeae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:41.869702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787badc-432e-4376-b154-7c0ce95082d8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:41.910169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:41.910800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.911110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.911357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.912071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:41.912372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:41.913786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 114/217] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ae9db-9f22-41d4-99e9-e0db73740c71 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ae9db-9f22-41d4-99e9-e0db73740c71 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ae9db-9f22-41d4-99e9-e0db73740c71 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 115/218] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:41.927650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:41.928192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:41.932954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/219] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:41.941437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:41.946386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:41.947942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:41.959768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:41.998787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.030776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 116/220] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:42.031787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.070806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029c5fe-a342-4b08-be3d-f1e49116a59f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.114324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.115056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.115683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.116109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] PATH_INFO: `/v3/users/f0e36e70cb214826860d428f1c3c8509` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.132362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] RBAC: Authorizing `identity:get_user(user_id=f0e36e70cb214826860d428f1c3c8509)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.133852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.135726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/221] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:41 2022] GET /identity/v3/users/f0e36e70cb214826860d428f1c3c8509 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:42.136345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.162501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54768560-82ac-42e6-ad3d-e655130fecfd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.208606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.209731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.210516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.211207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.226208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:42.231768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:42.233323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:42.234294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:42.236915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 117/222] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:42.238022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.244452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.271752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2e047-0bb4-4ebb-9cac-9967849b1028 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.275938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.316707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.317398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.317682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.317960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.331341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:42.336496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:42.337351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:42.338144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/223] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:42.338784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.380262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69156f2-4703-4100-ab17-1e3aa2d7a478 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.424356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.424976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.425259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.425580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.458692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5d634-1c62-4b9b-b235-598d5136dd60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.502951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.505069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.509162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.509802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.510087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.510459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.511251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:42.511557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.513079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.520040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.525258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/224] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:42.529913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.563707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.635551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 118/225] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:42.637087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.645245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307da7-4506-4723-82b4-06d853f20d99 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.681618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.682083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.682468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.682844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.683652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:42.684085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.686408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.696393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/226] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:42.697370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.754892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7433ad-2b10-4367-a4ab-2511ad7bc608 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.757588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.794191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.795716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.796027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:42.796356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.796866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] PATH_INFO: `/v3/users/9243e16a86944f888cf505df2df68396` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.798320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] RBAC: Authorizing `identity:get_user(user_id=9243e16a86944f888cf505df2df68396)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.800264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.802113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 119/227] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/users/9243e16a86944f888cf505df2df68396 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:42.803057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.814993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670fe4-e5db-49e1-9547-e184663da64a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.855700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.855700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.856090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.858575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.860923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.862519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.864417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/228] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:42.869488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.923245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8bed-a6ad-48b3-a652-f5280c98e488 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:42.960176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:42.961147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:42.961625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:42.962085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] PATH_INFO: `/v3/users/9243e16a86944f888cf505df2df68396` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:42.963410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] RBAC: Authorizing `identity:get_user(user_id=9243e16a86944f888cf505df2df68396)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:42.965104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:42.968241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 120/229] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/users/9243e16a86944f888cf505df2df68396 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:42.969223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:42.976297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3abe-3e44-4376-848f-c76b7713833e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:43.014531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:43.015199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:43.015529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:43.015841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:43.016525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:43.017294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:43.018621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:43.020617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/230] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:43.026430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:43.061430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:43.082487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19999a-8d59-4f0d-81a8-dcdf6cf1d589 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:43.122186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:43.125451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:43.125967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:43.126434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:43.127668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=f0e36e70cb214826860d428f1c3c8509) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:43.128198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:43.129864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:43.135946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 121/231] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:42 2022] GET /identity/v3/role_assignments?user.id=f0e36e70cb214826860d428f1c3c8509&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:43.140501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:43.149955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f586e-f4c5-4f69-80b1-2cd3e196c760 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:43.189882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:43.190452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:43.190734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:43.191077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:43.195627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:43.195627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:43.195627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/232] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:43 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:43.202105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:43.353678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:43.354867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e4833-8964-4d7f-a287-c73382bc2213 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:43.365394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc74e7-6614-431c-80dd-d5066accb35b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:43.388992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:43.393758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33275d-fedd-4a10-ad69-fe2d532b35b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:43.394650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33275d-fedd-4a10-ad69-fe2d532b35b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:43.395122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33275d-fedd-4a10-ad69-fe2d532b35b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:43.395678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33275d-fedd-4a10-ad69-fe2d532b35b2 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e/users/9243e16a86944f888cf505df2df68396/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:43.397999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33275d-fedd-4a10-ad69-fe2d532b35b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9670df9796144337814dd085d2d3de3e, user_id=9243e16a86944f888cf505df2df68396, role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:43.399656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33275d-fedd-4a10-ad69-fe2d532b35b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:43.408328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a66e6-b2b8-4d7c-bfca-760679bbd604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:43.409860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a66e6-b2b8-4d7c-bfca-760679bbd604 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:43.409860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a66e6-b2b8-4d7c-bfca-760679bbd604 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:43.409860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a66e6-b2b8-4d7c-bfca-760679bbd604 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e/users/9243e16a86944f888cf505df2df68396/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:43.411718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a66e6-b2b8-4d7c-bfca-760679bbd604 None admin] RBAC: Authorizing `identity:create_grant(project_id=9670df9796144337814dd085d2d3de3e, user_id=9243e16a86944f888cf505df2df68396, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:43.413205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a66e6-b2b8-4d7c-bfca-760679bbd604 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:43.420440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 122/233] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:43 2022] PUT /identity/v3/projects/9670df9796144337814dd085d2d3de3e/users/9243e16a86944f888cf505df2df68396/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:43.438500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/234] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:43 2022] PUT /identity/v3/projects/9670df9796144337814dd085d2d3de3e/users/9243e16a86944f888cf505df2df68396/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:49:44.096828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee28a12-e823-40fd-8313-b29cd75e29f0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.097822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee28a12-e823-40fd-8313-b29cd75e29f0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.098300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee28a12-e823-40fd-8313-b29cd75e29f0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.100199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 123/235] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:44.113376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.113842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.113842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.140263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:44.141475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:44.211956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74cdd0-ddef-4b4a-a06a-77776e610ab1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.212572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74cdd0-ddef-4b4a-a06a-77776e610ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.213447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74cdd0-ddef-4b4a-a06a-77776e610ab1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.215243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 124/236] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:44.225058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.225591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.226053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.235166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/237] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:44.245659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:44.247073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:44.279423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:44.316459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 125/238] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:44.378240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:44.404897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:44.445801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9768b-10a9-447c-87a2-75f792e1e5a2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:44.491166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:44.491356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:44.492296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.492666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.493067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.500337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:44.502225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:44.504112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02950f-a1c4-469c-bbba-cbef79e5c27e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:44.504299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:44.509567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:44.543154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:44.544208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.545306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.545769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.556874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:44.562054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:44.562924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:44.563724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:44.566440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 126/240] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:44.571925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:44.642407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2961e-cd21-4b9d-ac08-47b00f79b8e7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:44.702229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94677-1223-48d9-9309-9f014ffab567 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:44.708389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:44.709027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.709316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.709596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.710262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:44.711819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:44.743526 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:44.744478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/241] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:44 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:49:44.744722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.745326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.745787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.746647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:44.747150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:44.749527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:44.760858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 127/242] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:44.766871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:44.777003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:44.920460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55b5d6-1079-4de8-830e-2b7fff8f27c0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:44.960947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:44.961646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:44.961936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:44.962213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] PATH_INFO: `/v3/users/33b210bf459c49d29463e472ecd34517` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:44.978598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=33b210bf459c49d29463e472ecd34517)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:44.980111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:44.982770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/243] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:44 2022] GET /identity/v3/users/33b210bf459c49d29463e472ecd34517 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:44.988200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:45.011512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:45.104354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1ee6e-4f0a-4493-b609-e64e82fe45d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:45.146226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:45.146851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.147134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.147411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.226800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:45.228348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:45.249004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:45.347056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 128/244] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:45.352351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:45.429457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:45.472998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835d974-122d-4158-b10c-6c5f4cf6b7a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:45.515515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f92688-601d-460f-a49a-0a14a0c23deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:45.516110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f92688-601d-460f-a49a-0a14a0c23deb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.516407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f92688-601d-460f-a49a-0a14a0c23deb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.516673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f92688-601d-460f-a49a-0a14a0c23deb None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.517908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f92688-601d-460f-a49a-0a14a0c23deb None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:45.519344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f92688-601d-460f-a49a-0a14a0c23deb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:45.521349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/245] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:45 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:45.526857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:45.640829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c56a31-8e29-42e9-ae02-a713164081ae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:45.663080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:45.684595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:45.685528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.685857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.686131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.687173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] RBAC: Adding query filter params (role.id=7d1dedab3b264ed8816f26bb28b74647, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=33b210bf459c49d29463e472ecd34517) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:45.687487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:45.688970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba2c72-4362-42fd-8724-530a4ee5d3fa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:45.702615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 129/246] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:45 2022] GET /identity/v3/role_assignments?user.id=33b210bf459c49d29463e472ecd34517&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=7d1dedab3b264ed8816f26bb28b74647 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:45.871547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d71575-df45-4792-b87f-e88e47d14f8f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.871892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d71575-df45-4792-b87f-e88e47d14f8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.872160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d71575-df45-4792-b87f-e88e47d14f8f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.874152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/247] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:45.880148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.880437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.880719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.900590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:45.901922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:45.909828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c943000-c3bc-42eb-ad7a-bb60ecba14e7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.910119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c943000-c3bc-42eb-ad7a-bb60ecba14e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.910398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c943000-c3bc-42eb-ad7a-bb60ecba14e7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.911807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/248] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:45.917221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:45.917507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:45.917839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:45.937837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:45.939062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:46.014158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/249] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:46.038021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.042253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 130/250] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:46.116034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.120869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:46.162614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc502b8-4d62-46d1-81ac-123f5975919b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.198802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.199449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.199769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.200047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.211577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:46.216839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:46.217779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:46.218556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/251] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:46.224129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.328164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e6713-5455-4fe0-9603-61bb497cf932 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.353255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:46.361747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9d953-85d2-495a-9773-5754a3e8ada0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.388054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:46.399011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.399863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.400426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.400890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.401742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:46.402224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:46.404457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d6e47-296c-4c2a-9ffe-d6396ef7b404 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:46.413278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/252] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:46.414781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0efdb2-6133-4134-bec2-f1f9910bf144 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.415090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c1f72-63c8-4214-b9e6-ba4fdc570d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.415457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0efdb2-6133-4134-bec2-f1f9910bf144 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.415775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c1f72-63c8-4214-b9e6-ba4fdc570d88 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.416058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c1f72-63c8-4214-b9e6-ba4fdc570d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.416333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c1f72-63c8-4214-b9e6-ba4fdc570d88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.416506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0efdb2-6133-4134-bec2-f1f9910bf144 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.418673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/253] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:46.424037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0efdb2-6133-4134-bec2-f1f9910bf144 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.440662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c1f72-63c8-4214-b9e6-ba4fdc570d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2d8c1f72-63c8-4214-b9e6-ba4fdc570d88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:46.453663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:46.454536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:46.455342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 131/254] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:46.456418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.456718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.456964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.496582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:46.498078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:46.538015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0efdb2-6133-4134-bec2-f1f9910bf144 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.574609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.576303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.576993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.577480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] PATH_INFO: `/v3/users/9243e16a86944f888cf505df2df68396` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.578956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] RBAC: Authorizing `identity:get_user(user_id=9243e16a86944f888cf505df2df68396)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:46.580834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:46.583127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/255] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/users/9243e16a86944f888cf505df2df68396 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:46.584173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.625867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 132/256] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:46.626836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.656182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:46.719608 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b9de-a499-4079-912b-30c38d94a851 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.755983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442247-0b29-4229-a924-134276a74164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.756610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442247-0b29-4229-a924-134276a74164 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.756895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442247-0b29-4229-a924-134276a74164 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.757176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442247-0b29-4229-a924-134276a74164 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.757862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442247-0b29-4229-a924-134276a74164 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:46.758171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442247-0b29-4229-a924-134276a74164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:46.759587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442247-0b29-4229-a924-134276a74164 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:46.770876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/257] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:46.771847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442247-0b29-4229-a924-134276a74164 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:46.811232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:46.817817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126b111-ddd0-47d0-adc2-71b44771f6e3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.892942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.893898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.894350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.896004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.905544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442247-0b29-4229-a924-134276a74164 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:46.935989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:46.973793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:46.975062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:46.975347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:46.975634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:46.997163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:47.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:47.004017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:47.004814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/258] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:47.011585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.028959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.030506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.202019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:47.207860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 133/259] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:47.208766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.226461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03137-356e-4cad-b471-a27ac1617067 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.309056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.310135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.310868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.311363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] PATH_INFO: `/v3/users/f0e36e70cb214826860d428f1c3c8509` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.316371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] RBAC: Authorizing `identity:get_user(user_id=f0e36e70cb214826860d428f1c3c8509)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.318087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.319972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/260] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/users/f0e36e70cb214826860d428f1c3c8509 => generated 297 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:47.320939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.343639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb543-ab57-4ede-97b1-8a1a212533d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.385421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.386044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.386329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.386607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.387283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:47.387586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.394617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.410986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 134/261] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:47.411660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.414282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:47.483279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612577f7-c525-43d5-bdb6-5ef087b23d7f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.522457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.523108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.523438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.523750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.525366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.526950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.529481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/262] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:47.530408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.605359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:47.646803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5f72a-392d-49ca-9db3-e30eb228ba12 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.682960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.683778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.684069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.684349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] PATH_INFO: `/v3/users/9243e16a86944f888cf505df2df68396` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.685787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] RBAC: Authorizing `identity:get_user(user_id=9243e16a86944f888cf505df2df68396)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.687461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.689213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/263] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/users/9243e16a86944f888cf505df2df68396 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:47.689989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.699990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66e4b3-4518-4871-a766-c22f5d4ea567 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.704135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:47.787162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.788316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.788892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.789383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.790722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.792557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.795276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 135/264] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:47.801072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.805224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0accfd-86ac-47a5-87c6-1c37f279fb8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.845822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.846380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.846686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.846947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.847780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] RBAC: Adding query filter params (role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=9243e16a86944f888cf505df2df68396) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:47.848094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:47.849540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11cd56-0b30-4bda-8a96-11f0db7db28c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:47.862909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/265] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/role_assignments?user.id=9243e16a86944f888cf505df2df68396&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:47.864363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9eeb6-f22e-4155-b13f-0cab5d7ba342 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.864363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9eeb6-f22e-4155-b13f-0cab5d7ba342 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.864616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9eeb6-f22e-4155-b13f-0cab5d7ba342 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.876012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/266] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:47.876012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9eeb6-f22e-4155-b13f-0cab5d7ba342 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:47.926949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44b7aa-9560-4521-999d-bbfcbe6f3b41 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:47.969655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:47.983134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b622c76-27dd-488d-8ad5-6c163323e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:47.984163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b622c76-27dd-488d-8ad5-6c163323e21c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:47.984766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b622c76-27dd-488d-8ad5-6c163323e21c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:47.986393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b622c76-27dd-488d-8ad5-6c163323e21c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:47.992172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.091863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9eeb6-f22e-4155-b13f-0cab5d7ba342 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.108756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b622c76-27dd-488d-8ad5-6c163323e21c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.110853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b622c76-27dd-488d-8ad5-6c163323e21c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.135152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55ac3-a447-4607-876b-5c377559d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.135858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55ac3-a447-4607-876b-5c377559d936 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.136142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55ac3-a447-4607-876b-5c377559d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.136418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55ac3-a447-4607-876b-5c377559d936 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e/users/f0e36e70cb214826860d428f1c3c8509/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.138638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55ac3-a447-4607-876b-5c377559d936 None admin] RBAC: Authorizing `identity:create_grant(project_id=9670df9796144337814dd085d2d3de3e, user_id=f0e36e70cb214826860d428f1c3c8509, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.140143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55ac3-a447-4607-876b-5c377559d936 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.157566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/267] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:47 2022] PUT /identity/v3/projects/9670df9796144337814dd085d2d3de3e/users/f0e36e70cb214826860d428f1c3c8509/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:48.158875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652948-9e93-4aa5-971a-00a0e8c46210 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.159212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652948-9e93-4aa5-971a-00a0e8c46210 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.159496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652948-9e93-4aa5-971a-00a0e8c46210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.181798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e2652948-9e93-4aa5-971a-00a0e8c46210 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:48.183280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e2652948-9e93-4aa5-971a-00a0e8c46210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:48.257863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 136/268] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:48.263626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/269] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:48.264979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac12567-07ee-4a9d-9091-87a0c21d1f7c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.264979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac12567-07ee-4a9d-9091-87a0c21d1f7c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.264979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac12567-07ee-4a9d-9091-87a0c21d1f7c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.266299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 137/270] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:48.272306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.272657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.273046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.274576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac12567-07ee-4a9d-9091-87a0c21d1f7c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.304306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:48.304756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:48.322035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.337640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.390965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/271] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:48.395728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.418352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac12567-07ee-4a9d-9091-87a0c21d1f7c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.457249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f94de-74a6-493c-a81b-4140a51260bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.457893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f94de-74a6-493c-a81b-4140a51260bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.458176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f94de-74a6-493c-a81b-4140a51260bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.458453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f94de-74a6-493c-a81b-4140a51260bf None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.459559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f94de-74a6-493c-a81b-4140a51260bf None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.461052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f94de-74a6-493c-a81b-4140a51260bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.462891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 138/272] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:48.463981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46b0d3-1446-4fc8-ac7a-660ac5620e84 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.464324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46b0d3-1446-4fc8-ac7a-660ac5620e84 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.464604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46b0d3-1446-4fc8-ac7a-660ac5620e84 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.465928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 139/273] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:48.470553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46b0d3-1446-4fc8-ac7a-660ac5620e84 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.518116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ed34-265c-4f2c-913e-398373bb5706 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.553450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.554085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.554365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.554640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.565660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:48.570378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:48.571277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:48.572129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/274] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:48.573369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.584587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.590120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46b0d3-1446-4fc8-ac7a-660ac5620e84 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.599451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.629213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53559455-8d73-4a4f-8f12-1d0e26951ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.630208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53559455-8d73-4a4f-8f12-1d0e26951ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.630911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53559455-8d73-4a4f-8f12-1d0e26951ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.631350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53559455-8d73-4a4f-8f12-1d0e26951ad6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.632616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53559455-8d73-4a4f-8f12-1d0e26951ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.634745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53559455-8d73-4a4f-8f12-1d0e26951ad6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.637381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 140/275] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:48.639260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.639822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.640258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.659209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:48.660361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:48.688660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd5e20-e8e4-4170-aa05-f9499f638e13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.728094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.728697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.728975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.729242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.729957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=9243e16a86944f888cf505df2df68396) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:48.730260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.732307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.743829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 141/276] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:48.743829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.755970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/277] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/role_assignments?user.id=9243e16a86944f888cf505df2df68396&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:48.756905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.867989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.875864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b486f1-dd7a-4e65-a0c9-c81c09dcd92a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.889667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:48.906590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec260-12de-4f2a-8aa0-9bdc0684676f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:48.932875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.933738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.934022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.934299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.935694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:48.936114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.938445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.947508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:48.947738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 142/278] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:48.948506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:48.948642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:48.949210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:48.949628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:48.950581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:48.952841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:48.978060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/279] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:49:48 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:48.979235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:49.075397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c5c1-eb18-41d1-bd8a-abed37569076 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.130157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f9e45-6af9-477a-8bbf-7fd0581782a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.132461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.133382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.133885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.134356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.146681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:49.152551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:49.154039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:49.154841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:49.161893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 143/280] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:49.164065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:49.178553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.179212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.179493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.179826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] PATH_INFO: `/v3/users/127af4757fd143fcaf6558a30be0fb0b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.194921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:49.195447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:49.197007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] RBAC: Authorizing `identity:get_user(user_id=127af4757fd143fcaf6558a30be0fb0b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:49.199116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:49.201310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/281] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:48 2022] GET /identity/v3/users/127af4757fd143fcaf6558a30be0fb0b => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:49.206983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:49.297207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f17125-2b32-4469-a9de-c8b67c2a847f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.336457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e1b8f-72db-42c0-8521-4184026ddf47 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.342602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.343222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.343498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.343797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.344471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:49.345210 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:49.346610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:49.358745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 144/282] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:49.365011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:49.378498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.379526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.380053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.380561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.381545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, user.id=127af4757fd143fcaf6558a30be0fb0b, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:49.382106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:49.383836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:49.391440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/283] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 9 06:49:49 2022] GET /identity/v3/role_assignments?user.id=127af4757fd143fcaf6558a30be0fb0b&scope.system=all&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:49.475637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:49.497153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61aaec-39d5-47c0-ba6b-ac2d66b7bc0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.545385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.546042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.546356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.546598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] PATH_INFO: `/v3/users/33b210bf459c49d29463e472ecd34517` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.547785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] RBAC: Authorizing `identity:get_user(user_id=33b210bf459c49d29463e472ecd34517)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:49.549306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:49.551063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 145/284] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:49 2022] GET /identity/v3/users/33b210bf459c49d29463e472ecd34517 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:49.556828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:49.637033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:49.685365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affe632-faa5-4ee3-88eb-82f1402d489a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.721797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d4f1f-4a37-44a2-87ae-3a272e6aed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.722441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4f1f-4a37-44a2-87ae-3a272e6aed6e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.722720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4f1f-4a37-44a2-87ae-3a272e6aed6e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.722996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4f1f-4a37-44a2-87ae-3a272e6aed6e None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.724160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d4f1f-4a37-44a2-87ae-3a272e6aed6e None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:49.725821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d4f1f-4a37-44a2-87ae-3a272e6aed6e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:49.727814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/285] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:49 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:49.734388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:49.875488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cf06c-96c7-4089-988f-750d1fb5961c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:49.910430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842467-6cee-4187-bcb1-6ec69dd4c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:49.911100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842467-6cee-4187-bcb1-6ec69dd4c12d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:49.911380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842467-6cee-4187-bcb1-6ec69dd4c12d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:49.911682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842467-6cee-4187-bcb1-6ec69dd4c12d None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e/users/33b210bf459c49d29463e472ecd34517/roles/7d1dedab3b264ed8816f26bb28b74647` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:49.913591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842467-6cee-4187-bcb1-6ec69dd4c12d None admin] RBAC: Authorizing `identity:create_grant(project_id=9670df9796144337814dd085d2d3de3e, user_id=33b210bf459c49d29463e472ecd34517, role_id=7d1dedab3b264ed8816f26bb28b74647)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:49.915047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842467-6cee-4187-bcb1-6ec69dd4c12d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:49.929172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 146/286] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:49:49 2022] PUT /identity/v3/projects/9670df9796144337814dd085d2d3de3e/users/33b210bf459c49d29463e472ecd34517/roles/7d1dedab3b264ed8816f26bb28b74647 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:50.389618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f234548-9c71-4280-a990-acc296ac7312 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:50.389901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f234548-9c71-4280-a990-acc296ac7312 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:50.390179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f234548-9c71-4280-a990-acc296ac7312 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:50.391809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/287] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:50.397120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24524000-4d4c-4002-a22c-631d6214671f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:50.397551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24524000-4d4c-4002-a22c-631d6214671f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:50.397662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24524000-4d4c-4002-a22c-631d6214671f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:50.424575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-24524000-4d4c-4002-a22c-631d6214671f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:50.425789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-24524000-4d4c-4002-a22c-631d6214671f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:50.427865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:50.534298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 147/288] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:50.569025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f234548-9c71-4280-a990-acc296ac7312 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:50.733505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f234548-9c71-4280-a990-acc296ac7312 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:50.776043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:50.787120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:50.787800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:50.788084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:50.788359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:50.804732 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:50.809503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:50.810918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:50.811717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/289] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:50.817540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24524000-4d4c-4002-a22c-631d6214671f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:50.978294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24524000-4d4c-4002-a22c-631d6214671f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:51.022705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:51.023626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.024081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.024538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.025372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:51.025841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:51.027374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:51.035715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:51.041755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 148/290] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:51.047933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:51.196212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b02f-1044-4522-9193-7210737bc847 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:51.234792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf25f8-d11b-4e4e-b852-01311b7a0213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:51.235432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf25f8-d11b-4e4e-b852-01311b7a0213 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.235745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf25f8-d11b-4e4e-b852-01311b7a0213 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.236023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf25f8-d11b-4e4e-b852-01311b7a0213 None admin] PATH_INFO: `/v3/users/f0e36e70cb214826860d428f1c3c8509` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.237173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf25f8-d11b-4e4e-b852-01311b7a0213 None admin] RBAC: Authorizing `identity:get_user(user_id=f0e36e70cb214826860d428f1c3c8509)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:51.238631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf25f8-d11b-4e4e-b852-01311b7a0213 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:51.240591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/291] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:51 2022] GET /identity/v3/users/f0e36e70cb214826860d428f1c3c8509 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:51.246086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:51.348128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63363c5-6757-4d21-aad7-d2f3cd072fae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:51.382641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:51.383258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.383540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.383844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.469770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:51.476278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:51.477734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:51.546689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370423af-8f90-423d-8689-53997c4f5374 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.547036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370423af-8f90-423d-8689-53997c4f5374 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.547239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370423af-8f90-423d-8689-53997c4f5374 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.548757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/292] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:51.554333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.554611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.554887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.589632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:51.590860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:51.642175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 149/293] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:51.652671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:51.690820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/294] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:51.691364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:51.743702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:51.823052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:51.827275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c826a2f-6bb2-426e-8825-f76af37f1702 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:51.866986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf469b-1308-4e30-a3b1-f8c6a8b1e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:51.867647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf469b-1308-4e30-a3b1-f8c6a8b1e4ec None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.867937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf469b-1308-4e30-a3b1-f8c6a8b1e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.868212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf469b-1308-4e30-a3b1-f8c6a8b1e4ec None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.869341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf469b-1308-4e30-a3b1-f8c6a8b1e4ec None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:51.870769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf469b-1308-4e30-a3b1-f8c6a8b1e4ec None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:51.874897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f0698-fecc-439c-b9b9-f597ab841bc0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:51.878354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 150/295] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:51 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:51.879760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19b9d-b9cd-41fa-9648-eb05be664d8f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.880051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19b9d-b9cd-41fa-9648-eb05be664d8f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.880328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19b9d-b9cd-41fa-9648-eb05be664d8f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.881825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 151/296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:51.882539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19b9d-b9cd-41fa-9648-eb05be664d8f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:51.912719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c581e-6b05-4125-a823-51d6892e977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:51.913377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c581e-6b05-4125-a823-51d6892e977d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.913660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c581e-6b05-4125-a823-51d6892e977d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.913929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c581e-6b05-4125-a823-51d6892e977d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.925509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c581e-6b05-4125-a823-51d6892e977d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-964c581e-6b05-4125-a823-51d6892e977d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:51.930175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:51.931020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:51.931814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/297] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:51.933072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:51.935637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:51.935637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:51.952797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:51.953984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:51.998786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19b9d-b9cd-41fa-9648-eb05be664d8f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.003813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.027511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/298] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:52.035718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.056306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.057235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.057659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.058212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.059240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=f0e36e70cb214826860d428f1c3c8509) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:52.060485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.062446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.075254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 152/299] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:51 2022] GET /identity/v3/role_assignments?user.id=f0e36e70cb214826860d428f1c3c8509&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:52.076498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.152169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.172562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84b7f-6760-464a-b1ad-3a931b0dd981 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.209930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.210560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.210851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.211132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.211866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:52.212179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.213590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5ef8-90ca-4c9a-9b22-85de822ae472 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.223424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/300] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:52.229351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aee122-593b-47e0-a5e8-e605210a2485 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.229718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aee122-593b-47e0-a5e8-e605210a2485 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.230002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aee122-593b-47e0-a5e8-e605210a2485 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.231373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/301] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:52.237000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aee122-593b-47e0-a5e8-e605210a2485 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.251790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6eaec-852c-4a8f-8708-5b2982d96396 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.297648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca8f3-2357-4564-8f95-db0b78fd287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.298289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca8f3-2357-4564-8f95-db0b78fd287d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.298569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca8f3-2357-4564-8f95-db0b78fd287d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.298901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca8f3-2357-4564-8f95-db0b78fd287d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.300771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ca8f3-2357-4564-8f95-db0b78fd287d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.302025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ca8f3-2357-4564-8f95-db0b78fd287d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.303844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.304332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 153/302] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:52.305655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.305946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.306222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.327651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:52.329185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:52.378816 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aee122-593b-47e0-a5e8-e605210a2485 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.412191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 154/303] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:52.413019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.426704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.427948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.428897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.429680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] PATH_INFO: `/v3/users/b92be9ccc5ea40f4b38b8f2c641c72d6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.443876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.451294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] RBAC: Authorizing `identity:get_user(user_id=b92be9ccc5ea40f4b38b8f2c641c72d6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.452843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.454671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/304] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/users/b92be9ccc5ea40f4b38b8f2c641c72d6 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:52.455269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.524623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226b5f7-9a7d-488c-a180-cc5e1816b7ba None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.560571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.561181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.561458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.561726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] PATH_INFO: `/v3/groups` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.562331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.563864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.566501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.570104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aab03-8a78-4be3-ba1d-0b042433863e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.580760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 155/305] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Aug 9 06:49:52 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:49:52.581591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.622557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a1b3b-706f-4182-85e0-24392be41c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.624171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a1b3b-706f-4182-85e0-24392be41c91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.624658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a1b3b-706f-4182-85e0-24392be41c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.625192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a1b3b-706f-4182-85e0-24392be41c91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.638972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a1b3b-706f-4182-85e0-24392be41c91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-554a1b3b-706f-4182-85e0-24392be41c91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:52.644744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:52.645793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:52.646626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:52.649171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/306] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:52.650744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ac3fa-025a-47b1-b6da-f8cdf8a3bd8b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.651034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ac3fa-025a-47b1-b6da-f8cdf8a3bd8b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.651312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ac3fa-025a-47b1-b6da-f8cdf8a3bd8b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.653039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/307] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:52.656067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ac3fa-025a-47b1-b6da-f8cdf8a3bd8b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.705593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.718899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68c953-6029-4e19-a595-fc64eaa0bdcd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.758726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.759343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.759671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.759948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.760676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] RBAC: Adding query filter params (role.id=7d1dedab3b264ed8816f26bb28b74647, user.id=b92be9ccc5ea40f4b38b8f2c641c72d6, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:52.760986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.762421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290b2a6-ccd8-4156-a841-7f52453d3fe7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.768401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 156/308] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/role_assignments?user.id=b92be9ccc5ea40f4b38b8f2c641c72d6&scope.system=all&role.id=7d1dedab3b264ed8816f26bb28b74647 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:52.769761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.770050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.770325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.780208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ac3fa-025a-47b1-b6da-f8cdf8a3bd8b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:52.791064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:52.792338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:52.821328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:52.822346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:52.822832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:52.823249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:52.824250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:52.824709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:52.826483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:52.843390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/309] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:52.849020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.885104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:52.890371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 157/310] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:52.914295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:52.976043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8904-e1a3-4d2a-be03-538c22bd8970 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.014440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.015143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.015430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.015782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] PATH_INFO: `/v3/users/127af4757fd143fcaf6558a30be0fb0b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.016953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] RBAC: Authorizing `identity:get_user(user_id=127af4757fd143fcaf6558a30be0fb0b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:53.018871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:53.020982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/311] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/users/127af4757fd143fcaf6558a30be0fb0b => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:53.026505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:53.032295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:53.054285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48927-c75e-466b-9ce5-40504e69c275 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.107317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.107988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.108266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.108542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.123326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:53.129343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:53.130905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:53.131787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:53.132577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 158/312] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:53.139099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:53.144597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2da93e-5243-499c-aca4-9b412c935ad1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.181993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.182660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.182942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.183220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] PATH_INFO: `/v3/system/users/127af4757fd143fcaf6558a30be0fb0b/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.185283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=127af4757fd143fcaf6558a30be0fb0b, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:53.186815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:53.198262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/313] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Aug 9 06:49:53 2022] PUT /identity/v3/system/users/127af4757fd143fcaf6558a30be0fb0b/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:49:53.250370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1d10-0a5d-42be-ba76-1f2a8e00a84f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.289609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.290236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.290517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.290789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.291466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:53.291800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:53.293193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:53.307728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 159/314] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:53.309394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:53.423977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9f35-fd2e-4893-8c15-73a9dd6a0a65 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.459936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.460607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.460890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.461160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] PATH_INFO: `/v3/users/33b210bf459c49d29463e472ecd34517` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.462272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] RBAC: Authorizing `identity:get_user(user_id=33b210bf459c49d29463e472ecd34517)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:53.463730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:53.465720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/315] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:53 2022] GET /identity/v3/users/33b210bf459c49d29463e472ecd34517 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:53.471690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:53.481163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:53.578819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274a37-e11e-4bbd-9d15-d849fd58af49 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.613329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.613947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.614236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.614516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.684175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:53.686297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:53.694238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:53.795857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 160/316] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:53.800690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:53.828303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:53.912257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c6a5a-6306-442e-b252-aff8d2ccc0eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:53.948790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25f553-d041-484d-9150-6b8030768fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:53.949439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f553-d041-484d-9150-6b8030768fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:53.949723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f553-d041-484d-9150-6b8030768fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:53.950000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f553-d041-484d-9150-6b8030768fb3 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:53.951237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25f553-d041-484d-9150-6b8030768fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:53.952706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25f553-d041-484d-9150-6b8030768fb3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:53.954669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/317] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:53 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:53.960412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:54.064115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b716b-e723-4792-9c21-6c3bae2482c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:54.071699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:54.105940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:54.106870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:54.107346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:54.107847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:54.108769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] RBAC: Adding query filter params (role.id=7d1dedab3b264ed8816f26bb28b74647, scope.project.id=9670df9796144337814dd085d2d3de3e, user.id=33b210bf459c49d29463e472ecd34517) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:54.109292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:54.110930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a50c40-bc11-412c-9c62-e695e390b5bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:54.117555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 161/318] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:49:53 2022] GET /identity/v3/role_assignments?user.id=33b210bf459c49d29463e472ecd34517&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=7d1dedab3b264ed8816f26bb28b74647 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:54.721421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0152-a61f-4962-889d-bb72adc7f53f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:54.721801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0152-a61f-4962-889d-bb72adc7f53f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:54.722000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0152-a61f-4962-889d-bb72adc7f53f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:54.723347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/319] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:54.728370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3434-7b91-4660-96aa-c7deffdeb4eb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:54.728803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3434-7b91-4660-96aa-c7deffdeb4eb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:54.728960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3434-7b91-4660-96aa-c7deffdeb4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:54.747213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5eaa3434-7b91-4660-96aa-c7deffdeb4eb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:54.748571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5eaa3434-7b91-4660-96aa-c7deffdeb4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:54.817014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 162/320] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:54.840220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0152-a61f-4962-889d-bb72adc7f53f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:54.861730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81c40f-46fd-4394-a77b-1ced41524865 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:54.861881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81c40f-46fd-4394-a77b-1ced41524865 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:54.862042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81c40f-46fd-4394-a77b-1ced41524865 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:54.863532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 163/321] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:54.868141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:54.868430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:54.868701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:54.887130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:54.888909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:54.940581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:54.953052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0152-a61f-4962-889d-bb72adc7f53f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:54.960364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 164/322] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:49:54.983879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:54.992637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:54.993275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:54.993556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:54.993831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:54.994925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:54.996394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:54.998758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/323] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:55.004462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.014631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:55.089218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60fc5e-4460-4a27-9a12-c5ab4aefcfe7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.110765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fca550-0bfe-4d28-9b13-2e13aa073fcc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.125252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.125883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.126158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.126420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.138151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:55.143349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:55.144271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:55.145038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 165/324] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:55.150056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.152115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16be80-dded-4479-b916-dedad669c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.152735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16be80-dded-4479-b916-dedad669c819 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.153019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16be80-dded-4479-b916-dedad669c819 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.153942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16be80-dded-4479-b916-dedad669c819 None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.154493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16be80-dded-4479-b916-dedad669c819 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:55.156064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16be80-dded-4479-b916-dedad669c819 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:55.176327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/325] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Aug 9 06:49:55 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:49:55.256704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:55.269587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5c8b-009c-4e45-b1f3-ed6dccf9394e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.308350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.308964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368dbe3-3f14-43c6-b29c-aa2f8da914f4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.309259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368dbe3-3f14-43c6-b29c-aa2f8da914f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.310076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368dbe3-3f14-43c6-b29c-aa2f8da914f4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.310550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.311065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.311528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.311660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/326] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:49:55.312663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:55.313162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:55.314736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:55.317026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.317318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.317599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.328596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 166/327] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:55.334341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.336469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:55.337690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:55.354564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:55.418396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/328] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:55.442407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.456391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701207b-7d66-4044-b886-39d95cfa013a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.494967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.495656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.495944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.496424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] PATH_INFO: `/v3/users/b92be9ccc5ea40f4b38b8f2c641c72d6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.499079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] RBAC: Authorizing `identity:get_user(user_id=b92be9ccc5ea40f4b38b8f2c641c72d6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:55.500650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:55.502442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 167/329] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:55 2022] GET /identity/v3/users/b92be9ccc5ea40f4b38b8f2c641c72d6 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:55.509527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.532327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:55.584473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebff51-5373-4cda-95e7-f76c8b159919 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.636300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.637017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.637281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.637609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.645872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049ea42-0f19-47d3-959d-165dd5b6fad6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.650942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:55.656942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:55.657861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:55.658657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/330] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:55.663864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.677114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:55.694405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.695977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.696403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.697048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] PATH_INFO: `/v3/system/users/b92be9ccc5ea40f4b38b8f2c641c72d6/roles/7d1dedab3b264ed8816f26bb28b74647` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.699346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b92be9ccc5ea40f4b38b8f2c641c72d6, role_id=7d1dedab3b264ed8816f26bb28b74647)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:55.701732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:55.716847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 168/331] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Aug 9 06:49:55 2022] PUT /identity/v3/system/users/b92be9ccc5ea40f4b38b8f2c641c72d6/roles/7d1dedab3b264ed8816f26bb28b74647 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:49:55.799563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760f474-3864-4b99-9848-e8b11f762453 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:55.846395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:55.847009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:55.847288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:55.847557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:55.848260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:55.848558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:55.850012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:55.860020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/332] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:49:55.865556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:55.975147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:55.995209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8bc8f-32cf-4d78-a478-7a9cc36a731d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:56.038794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10654d-202a-4021-a830-79481ea7e8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:56.039815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10654d-202a-4021-a830-79481ea7e8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:56.040298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10654d-202a-4021-a830-79481ea7e8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:56.040767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10654d-202a-4021-a830-79481ea7e8a9 None admin] PATH_INFO: `/v3/users/127af4757fd143fcaf6558a30be0fb0b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:56.042167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10654d-202a-4021-a830-79481ea7e8a9 None admin] RBAC: Authorizing `identity:get_user(user_id=127af4757fd143fcaf6558a30be0fb0b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:56.043856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10654d-202a-4021-a830-79481ea7e8a9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:56.046401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 169/333] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:49:55 2022] GET /identity/v3/users/127af4757fd143fcaf6558a30be0fb0b => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:56.052449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:56.145976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:56.187796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2d6d-5bb4-4ec1-ab64-b9c707d114a1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:56.225778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:56.226384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:56.226662 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:56.226935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:56.227693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] RBAC: Adding query filter params (role.id=036ddec2546a4fe7bd3b8eca42111ba0, user.id=127af4757fd143fcaf6558a30be0fb0b, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:56.228000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:56.229421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a60b9-34ff-4d2c-a7c4-f18e3f5f8ba1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:56.243821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/334] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 9 06:49:56 2022] GET /identity/v3/role_assignments?user.id=127af4757fd143fcaf6558a30be0fb0b&scope.system=all&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:57.523535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b64e1-93a6-4329-8aa5-6e89a05d371e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:57.524618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b64e1-93a6-4329-8aa5-6e89a05d371e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:57.525098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b64e1-93a6-4329-8aa5-6e89a05d371e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:57.526689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 170/335] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:57.532693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a5cc-b0e2-4f6b-95ca-e542edb82f0d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:57.533581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a5cc-b0e2-4f6b-95ca-e542edb82f0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:57.534078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a5cc-b0e2-4f6b-95ca-e542edb82f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:57.561300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53f2a5cc-b0e2-4f6b-95ca-e542edb82f0d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:57.562975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53f2a5cc-b0e2-4f6b-95ca-e542edb82f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:57.655257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/336] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:57.691424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b64e1-93a6-4329-8aa5-6e89a05d371e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:57.713977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bda733-d069-4a70-9037-917b58804c7b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:57.714686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bda733-d069-4a70-9037-917b58804c7b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:57.715179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bda733-d069-4a70-9037-917b58804c7b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:57.717803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/337] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:57.727224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:57.727843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:57.728695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:57.767156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:57.768722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:57.880544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:57.909851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/338] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 06:49:57.921246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b64e1-93a6-4329-8aa5-6e89a05d371e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:57.965182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:57.983123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:57.983799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:57.984085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:57.984360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.011374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:58.016991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:58.017882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:58.019099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 171/339] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:58.026082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:58.058580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:58.152757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdae86-b0f3-4a66-bbbf-79759b85bb26 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:58.213710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:58.214358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.214639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.214910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.232972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:58.237719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:58.238615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:58.239842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/340] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:49:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:49:58.246413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:58.296369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924078ee-3e9f-4607-889d-30729aff4499 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:58.315965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:58.351000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:58.351676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.351961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.352232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.352913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:58.353213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:58.354672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:58.368288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 172/341] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:58.374768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:58.401917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7806e7-2caa-429c-82d1-1c8fb94385f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:58.442472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:58.443004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.447149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.447149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.447149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:58.447149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:58.447149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedd460-cf61-44be-ae47-dc0b48183fe9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:58.456936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/342] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:49:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:58.458284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6aaa37-7ecd-42ec-88c1-9545466eb6c5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.458571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6aaa37-7ecd-42ec-88c1-9545466eb6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.458871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6aaa37-7ecd-42ec-88c1-9545466eb6c5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.460485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/343] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:58.461042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6aaa37-7ecd-42ec-88c1-9545466eb6c5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:58.524020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:58.571761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22ac31-08ee-4cd6-aa3d-83f936b18a94 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:58.579188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6aaa37-7ecd-42ec-88c1-9545466eb6c5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:58.590384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:58.615629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76db3d-6850-42a4-a3c1-3bad46aa6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:58.615827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76db3d-6850-42a4-a3c1-3bad46aa6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.616058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76db3d-6850-42a4-a3c1-3bad46aa6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.616227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76db3d-6850-42a4-a3c1-3bad46aa6cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.636499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230c151-8677-45be-b191-c0be15c9b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:58.637469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230c151-8677-45be-b191-c0be15c9b2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.637996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230c151-8677-45be-b191-c0be15c9b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.638555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230c151-8677-45be-b191-c0be15c9b2c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.688927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76db3d-6850-42a4-a3c1-3bad46aa6cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:58.690490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76db3d-6850-42a4-a3c1-3bad46aa6cd4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:58.795503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/344] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:58.796127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230c151-8677-45be-b191-c0be15c9b2c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:58.796796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.797082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.797352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.797619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230c151-8677-45be-b191-c0be15c9b2c9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:58.815331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:58.816610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:58.864011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:58.908599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/345] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:58.909451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:58.959625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:58.991675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 173/346] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:49:58.993163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe337b-8aa2-42eb-88ae-55d659c14b85 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:58.993572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe337b-8aa2-42eb-88ae-55d659c14b85 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:58.993996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe337b-8aa2-42eb-88ae-55d659c14b85 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:58.995711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 174/347] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:49:58.996581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe337b-8aa2-42eb-88ae-55d659c14b85 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.062846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09e014-4b2f-4817-afa5-d68f460824c2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.107734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a102bfb-9942-4b0a-a114-fbbcb6c62110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.108393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a102bfb-9942-4b0a-a114-fbbcb6c62110 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.108839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a102bfb-9942-4b0a-a114-fbbcb6c62110 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.109898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a102bfb-9942-4b0a-a114-fbbcb6c62110 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.111642 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a102bfb-9942-4b0a-a114-fbbcb6c62110 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:59.113785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a102bfb-9942-4b0a-a114-fbbcb6c62110 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:59.116141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/348] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:58 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:59.117933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.118363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.118781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.130879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe337b-8aa2-42eb-88ae-55d659c14b85 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.140839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:49:59.142760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:49:59.152951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:59.172001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.173853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.175358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.176061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.190671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:59.196864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:59.197800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:49:59.198599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:49:59.201531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 175/349] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:59.202530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.234592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/350] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:49:59.235844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.246242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:59.322160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf1cdf-c3f8-44e6-ac77-cafbdd7afd56 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.361475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.362511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.363012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.363493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.366910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:59.368699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:59.370844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 176/351] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:59.372576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.377052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8e4fd-59ac-4ede-a05a-54f92b7dc67c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.420917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.421925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.422466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.422960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.430261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:59.432117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:59.434133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/352] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:59.440755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.445118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:59.508283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8684e-ef58-4497-a58d-d99af48d729d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.510153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:59.558744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.559357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.560259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.560482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.560943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de280b-c5bc-4a62-9ae3-3977c90d1f89 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.561167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:59.561472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:59.562875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:59.576011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 177/353] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:59.577351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.614626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.615260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.615556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.615844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.630097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:49:59.636617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:49:59.637499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:49:59.638311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/354] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:49:59.639446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.708815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370231-1852-44f4-82f6-39e88c7e05d2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.724040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:59.758655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.759296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.759642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.759929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.761520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:59.764047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:59.766846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 178/355] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:59.766846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.776981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df574-3686-495a-b73d-1040c4f4999c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.808776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:49:59.835487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.836627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.836908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.837180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.837921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=ec2591650a2e438db15b3fccbca1ca57) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:49:59.838231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:49:59.839731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:49:59.851688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/356] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:49:59.851688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:49:59.906047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a37d7-0f2f-4c85-b14e-737ed121c686 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:49:59.961109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80cc4c-8871-4f0b-989a-b9919ed3ae08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:49:59.962305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80cc4c-8871-4f0b-989a-b9919ed3ae08 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:49:59.962762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80cc4c-8871-4f0b-989a-b9919ed3ae08 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:49:59.963684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80cc4c-8871-4f0b-989a-b9919ed3ae08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:49:59.976684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d8ce-29d4-45cc-8c42-1a489c0d749f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.009897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.021479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.022245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.022535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.022821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.023510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:00.023849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.025384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05a3d7-0ae0-46b0-91de-b9f64dda81a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.034959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/357] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:00.036694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479af78-1298-48e1-b0f9-67fe52087464 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.037298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479af78-1298-48e1-b0f9-67fe52087464 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.037776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479af78-1298-48e1-b0f9-67fe52087464 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.043574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/358] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:00.045482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479af78-1298-48e1-b0f9-67fe52087464 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.063984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80cc4c-8871-4f0b-989a-b9919ed3ae08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.065454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80cc4c-8871-4f0b-989a-b9919ed3ae08 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.111658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.191821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 179/359] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:00.193103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd15b2-92dd-4dda-8538-50f76c3fcfff None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.193103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd15b2-92dd-4dda-8538-50f76c3fcfff None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.193103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd15b2-92dd-4dda-8538-50f76c3fcfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.195925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479af78-1298-48e1-b0f9-67fe52087464 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.214706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2cdd15b2-92dd-4dda-8538-50f76c3fcfff None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:00.216156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2cdd15b2-92dd-4dda-8538-50f76c3fcfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:00.245407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.246357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.246644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.246973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.248297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=ec2591650a2e438db15b3fccbca1ca57) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:00.248611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.250067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.257956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/360] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:00.257956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.307459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 180/361] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:00.308816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3990b-a3b5-46a2-83c8-604996122b3d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.309176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3990b-a3b5-46a2-83c8-604996122b3d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.309517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3990b-a3b5-46a2-83c8-604996122b3d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.311779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 181/362] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:00.317376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3990b-a3b5-46a2-83c8-604996122b3d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.335872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.408768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1b213-2171-43c9-bdc3-0836d1eb047d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.431253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.458158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954edf7-d2f6-4c47-92e5-1377f36ccf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.458896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954edf7-d2f6-4c47-92e5-1377f36ccf82 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.459185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954edf7-d2f6-4c47-92e5-1377f36ccf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.459467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954edf7-d2f6-4c47-92e5-1377f36ccf82 None admin] PATH_INFO: `/v3/users/b92be9ccc5ea40f4b38b8f2c641c72d6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.460661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954edf7-d2f6-4c47-92e5-1377f36ccf82 None admin] RBAC: Authorizing `identity:get_user(user_id=b92be9ccc5ea40f4b38b8f2c641c72d6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.462248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954edf7-d2f6-4c47-92e5-1377f36ccf82 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.464993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/363] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/users/b92be9ccc5ea40f4b38b8f2c641c72d6 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:00.466373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.466669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.466945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.476538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3990b-a3b5-46a2-83c8-604996122b3d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.489565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:00.490774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:00.519591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.520522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.520813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.521088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.522848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.524902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.527669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 182/364] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:00.529154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.579408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/365] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:00.580693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.660087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.672527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566596f-e87e-455d-8986-481efdd2da9b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.709489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893937-7847-40ac-9a60-4ec40f792f6f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.710130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.718354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.718995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.719272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.719668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.732097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:00.737171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:00.738071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:00.738862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 183/366] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:00.739517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.755744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.756422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.756807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.757088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.758041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] RBAC: Adding query filter params (role.id=7d1dedab3b264ed8816f26bb28b74647, user.id=b92be9ccc5ea40f4b38b8f2c641c72d6, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:00.758355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.759963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.774838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/367] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/role_assignments?user.id=b92be9ccc5ea40f4b38b8f2c641c72d6&scope.system=all&role.id=7d1dedab3b264ed8816f26bb28b74647 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:00.775763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.853238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4aad9-64ab-49a9-baa0-6a81708d8578 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.890991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.891760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.892047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.892321 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.893474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:00.894699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4988-c400-46ec-ba5a-147e99e84693 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:00.894924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:00.896851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 184/368] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:00.897526 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.909758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:00.940241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:00.940925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:00.941211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:00.941482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:00.954148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:00.960186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:00.961119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:00.962330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/369] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:00.962972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:00.969110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:01.017654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2065e-0fe3-47b8-8751-1a5eed1ef97d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.055431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:01.060291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:01.060885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:01.061360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:01.062283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:01.062782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:01.064396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:01.085886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 185/370] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:01.086190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962586-b9be-4ca0-b34d-4d68e273b5d4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.087258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:01.131953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:01.132567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:01.132852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:01.133126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:01.133947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=9670df9796144337814dd085d2d3de3e) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:01.134261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:01.135906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:01.143539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/371] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:00 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:01.144789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:01.214166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:01.235068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc4fa-4e7a-4e85-a76b-fdef373955d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.260857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:01.281351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:01.282158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:01.282447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:01.282724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:01.283420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:01.283764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:01.285188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:01.296388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 186/372] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:01.302329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:01.388200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ab5f4-4496-4788-9223-422b00799b1e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.451859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72fd8b-78d4-4897-9a65-097e709ac77c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.467717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:01.468324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:01.468606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:01.468881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:01.493412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:01.494026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:01.494307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:01.494578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:01.556756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:01.558350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:01.568178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:01.589941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:01.592778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:01.595638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:01.677564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/373] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:01.683188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:01.778569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 187/374] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:50:01.784188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:01.795639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de66db-3a6c-48fd-8f94-b6e0e5b0f838 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.828472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:01.835113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:01.838584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:01.838584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:01.838584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:01.838584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:01.842019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:01.844009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/375] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:01 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:01.849790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:01.969663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf7615-dbaf-42c6-85f9-c615b9123431 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:01.988549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:02.010042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:02.011035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.011545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.012201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.014350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:02.016052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:02.018586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/376] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:01 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:02.024603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:02.037891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac23bd-48ab-4757-942c-5e04c84de9f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:02.105284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:02.119255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:02.120312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.120874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.121354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] PATH_INFO: `/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.122788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] RBAC: Authorizing `identity:get_project(project_id=489e2c3ffdae42c6a83d2b2655912ea9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:02.131617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:02.133812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 188/377] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:01 2022] GET /identity/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9 => generated 344 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:02.140558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:02.143285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebdbb9-2e18-4100-80b5-7e8a193c9c06 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:02.186904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:02.187529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.187845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.188124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.189484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=9670df9796144337814dd085d2d3de3e) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:02.189853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:02.191459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:02.198678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/378] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:02 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:02.309425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea7f3b-880d-46f6-9754-16a8a119fb31 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:02.360520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b617336-9cf2-4a71-b2a3-5364d246a49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:02.361181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b617336-9cf2-4a71-b2a3-5364d246a49b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.361460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b617336-9cf2-4a71-b2a3-5364d246a49b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.361734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b617336-9cf2-4a71-b2a3-5364d246a49b None admin] PATH_INFO: `/v3/groups/da4d5556f9e140e488b15d4f056211ff` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.374438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b617336-9cf2-4a71-b2a3-5364d246a49b None admin] RBAC: Authorizing `identity:get_group(group_id=da4d5556f9e140e488b15d4f056211ff)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:02.375929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b617336-9cf2-4a71-b2a3-5364d246a49b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:02.378248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 189/379] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:02 2022] GET /identity/v3/groups/da4d5556f9e140e488b15d4f056211ff => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:02.384263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:02.507968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fe1a7-8f9a-4d7d-8f7f-3f2cf97f37f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:02.520124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57248e-1c11-41d3-b0a9-5a180bdef21c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.520420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57248e-1c11-41d3-b0a9-5a180bdef21c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.520675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57248e-1c11-41d3-b0a9-5a180bdef21c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.522015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 190/380] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:02.529086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d5073-aed1-44fe-ab92-4b904ba79b5b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.529372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d5073-aed1-44fe-ab92-4b904ba79b5b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.529650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d5073-aed1-44fe-ab92-4b904ba79b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.547575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:02.548581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.549083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.549572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.550501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] RBAC: Adding query filter params (group.id=da4d5556f9e140e488b15d4f056211ff, role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=489e2c3ffdae42c6a83d2b2655912ea9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:02.551522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:02.553223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:02.560460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/381] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:02 2022] GET /identity/v3/role_assignments?group.id=da4d5556f9e140e488b15d4f056211ff&scope.project.id=489e2c3ffdae42c6a83d2b2655912ea9&role.id=d58e69f2b822466daca66a9696521a24 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:02.561279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:02.567877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6d7d5073-aed1-44fe-ab92-4b904ba79b5b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:02.569120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6d7d5073-aed1-44fe-ab92-4b904ba79b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:02.585396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:02.669509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 191/382] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:02.704376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:02.831456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18334dd9-3897-4cf4-8082-c9caafa96e54 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:02.876417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:02.877706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.878202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.878680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.892037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:02.892344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffc9f-3c9d-497b-b099-facbe79f5001 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.892783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffc9f-3c9d-497b-b099-facbe79f5001 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.892944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffc9f-3c9d-497b-b099-facbe79f5001 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.894600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 192/383] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:02.898669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:02.899570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:02.900426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:02.902937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/384] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:02.906527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:02.906778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:02.907057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:02.908526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:02.929272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:02.930487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:03.019691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 193/385] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:03.024830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:03.050981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.109660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec2c92-6599-4a84-9577-e40e85dfdce8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:03.174952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:03.175654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:03.175942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:03.176212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:03.176899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:03.177200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:03.178590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:03.199425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:03.203330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/386] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:03.212606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.233744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507071c-7ffb-4351-aa1d-3ae68ff778dc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:03.282760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:03.283564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:03.284069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:03.284512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:03.296908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:03.302368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:03.304141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 194/387] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:03.316684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.451709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1451c-d483-418d-8531-ba697f446c36 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:03.480622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:03.510552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fafc-0713-40a5-a482-3a4647eb7a23 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:03.520678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:03.521350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:03.521625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:03.521907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:03.529002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:03.530471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:03.532344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/388] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:03 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:03.537759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.569064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:03.570086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:03.570556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:03.571027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:03.572246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:03.572748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:03.575545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:03.578403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:03.592765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 195/389] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:50:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:03.604257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.779296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d507-fc56-4dc7-b3e2-d79f9cdc0eb8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:03.822268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:03.823236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:03.824725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:03.825754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:03.828585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:03.831813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:03.839860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 196/390] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:03 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:03.843697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.845233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd7b10-acb4-4177-b210-be6d0a4ad8b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:03.901983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:03.903043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:03.904379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:03.904875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:03.907260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:03.909041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:03.912184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/391] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:03 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:03.921075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:03.968451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:04.012166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c42dd1-3dd7-42f0-be9e-765184169403 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:04.041751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:04.051219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:04.052329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.052778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.053207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.054484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:04.056273 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:04.058148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 197/392] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:03 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:04.064980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:04.145472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf08392-6e5f-4eef-bbed-4cb990cccc02 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:04.188921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9881f-4c59-478d-b5f2-b6561bb1912f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:04.189574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345db2dd-076c-4ccf-9e62-c0f5340d03a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:04.190260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db2dd-076c-4ccf-9e62-c0f5340d03a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.190550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db2dd-076c-4ccf-9e62-c0f5340d03a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.190835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db2dd-076c-4ccf-9e62-c0f5340d03a2 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57/groups/312a36e1e62642d0b76a62f93979379e/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.193110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345db2dd-076c-4ccf-9e62-c0f5340d03a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec2591650a2e438db15b3fccbca1ca57, group_id=312a36e1e62642d0b76a62f93979379e, role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:04.194857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345db2dd-076c-4ccf-9e62-c0f5340d03a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:04.213058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/393] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 9 06:50:03 2022] PUT /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57/groups/312a36e1e62642d0b76a62f93979379e/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:50:04.214356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7eed3-b356-4715-b7f0-6990a802ce7f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.214652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7eed3-b356-4715-b7f0-6990a802ce7f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.214940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7eed3-b356-4715-b7f0-6990a802ce7f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.220821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/394] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:04.229047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.229392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.229654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.242341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:04.256801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:04.257550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.257919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.258195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57/groups/312a36e1e62642d0b76a62f93979379e/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.260538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] RBAC: Authorizing `identity:create_grant(project_id=ec2591650a2e438db15b3fccbca1ca57, group_id=312a36e1e62642d0b76a62f93979379e, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:04.262250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:04.269812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:04.271090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:04.289977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 198/395] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 9 06:50:04 2022] PUT /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57/groups/312a36e1e62642d0b76a62f93979379e/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:04.357693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/396] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:04.391414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:04.516351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fbce-3998-4f72-b890-cb533184844e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:04.565105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:04.566051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.566508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.566956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.578729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:04.584038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:04.584883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:04.585714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 199/397] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:04.592389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:04.647676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:04.706963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215be1-aea6-4249-84e7-006729c67be8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:04.746639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:04.747804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.748282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.748745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.749621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:04.750111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:04.751704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:04.767017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/398] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:04.772628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:04.856157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:04.916746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23862-27b4-4dea-9bc0-c7f32545bc4c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:04.962534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee887f46-83e0-4092-a304-4d866c4d1562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:04.963233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee887f46-83e0-4092-a304-4d866c4d1562 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:04.963520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee887f46-83e0-4092-a304-4d866c4d1562 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:04.963821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee887f46-83e0-4092-a304-4d866c4d1562 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:04.968304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee887f46-83e0-4092-a304-4d866c4d1562 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:04.969777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee887f46-83e0-4092-a304-4d866c4d1562 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:04.971720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 200/399] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:04 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:04.977143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.005314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d31bb9-39ba-4bce-853a-9eac51884b02 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.005661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d31bb9-39ba-4bce-853a-9eac51884b02 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.005875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d31bb9-39ba-4bce-853a-9eac51884b02 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.007656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 201/400] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:50:05.018242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.018661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.019039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.041829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:05.043048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:05.132945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 202/401] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:05.166238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.182765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174958d-9c50-4e8a-ba3c-ad730c710147 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.190940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:05.253571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:05.254219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.254504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.254780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.256726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:05.258171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:05.259826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/402] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:04 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:05.266279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.266803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:05.329166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59256876-4d00-4f0e-a1c9-1993d0d8d016 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.373189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:05.374134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.374414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.374686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.401914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:05.407311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:05.408196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:05.408951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:05.417275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 203/403] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:05.418112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.479147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8997e-eb4c-421f-a95e-352fc73ba49c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.518480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5345dd-3148-478e-95f3-2c7152c8bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:05.519154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5345dd-3148-478e-95f3-2c7152c8bbf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.519439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5345dd-3148-478e-95f3-2c7152c8bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.519743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5345dd-3148-478e-95f3-2c7152c8bbf2 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e/groups/312a36e1e62642d0b76a62f93979379e/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.521675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5345dd-3148-478e-95f3-2c7152c8bbf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9670df9796144337814dd085d2d3de3e, group_id=312a36e1e62642d0b76a62f93979379e, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:05.523111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5345dd-3148-478e-95f3-2c7152c8bbf2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:05.541270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/404] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 9 06:50:05 2022] PUT /identity/v3/projects/9670df9796144337814dd085d2d3de3e/groups/312a36e1e62642d0b76a62f93979379e/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:50:05.542773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75782e1-419c-40ae-b51b-21d5c87940a3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.542966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75782e1-419c-40ae-b51b-21d5c87940a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.543238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75782e1-419c-40ae-b51b-21d5c87940a3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.545633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/405] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:05.551836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.552161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.552394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.557169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6520764-bcec-4ec7-91c5-6d5ee7da9957 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.570485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:05.575851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:05.577235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:05.604996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:05.605983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.607080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.607640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.608607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:05.609216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:05.610799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:05.614280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:05.621566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 204/406] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:05.627788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.679075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/407] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:05.708191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.753191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04596e1-0867-44c7-9d3b-38cf1d2f7d50 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.806589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:05.807377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.808185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.808185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.810329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:05.811991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:05.814364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 205/408] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:05 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:05.819527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.873582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70e407-19c4-4943-9202-de1f05d2d09b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.886005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:05.921604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:05.923310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:05.924123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:05.924639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:05.939114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:05.946336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:05.947283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:05.948099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:05.951142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/409] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:05.957132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:05.979974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3ce71-fec4-420c-aca2-a3ca4eb10072 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:05.993250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:06.078280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:06.078932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.079152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.079423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.080585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:06.082032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:06.083701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89b3fb-a438-45a0-b48e-e48b9e4f23f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:06.084044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 206/410] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:05 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:06.092332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:06.132967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:06.133991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.134494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.134960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.136448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:06.136951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:06.139636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:06.155778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/411] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:06.164654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:06.212413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e172-9061-4ef6-9859-efd4fa7fb604 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:06.222751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:06.253268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:06.254483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.254973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.255440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e/groups/312a36e1e62642d0b76a62f93979379e/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.258106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9670df9796144337814dd085d2d3de3e, group_id=312a36e1e62642d0b76a62f93979379e, role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:06.260186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:06.283498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 207/412] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 9 06:50:06 2022] PUT /identity/v3/projects/9670df9796144337814dd085d2d3de3e/groups/312a36e1e62642d0b76a62f93979379e/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:50:06.335715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:06.352551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096f68-256f-46e8-b24e-a1f74938f419 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:06.390860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:06.391835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.392263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.392740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] PATH_INFO: `/v3/groups/da4d5556f9e140e488b15d4f056211ff` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.394095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] RBAC: Authorizing `identity:get_group(group_id=da4d5556f9e140e488b15d4f056211ff)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:06.395793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:06.402153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/413] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:06 2022] GET /identity/v3/groups/da4d5556f9e140e488b15d4f056211ff => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:06.403798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:06.523623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76b229-38e3-400b-a41b-3f55b88b79c0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:06.560368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5a18d-0ede-46c8-9643-f05c845accd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:06.561004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5a18d-0ede-46c8-9643-f05c845accd8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.561289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5a18d-0ede-46c8-9643-f05c845accd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.561562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5a18d-0ede-46c8-9643-f05c845accd8 None admin] PATH_INFO: `/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.562708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5a18d-0ede-46c8-9643-f05c845accd8 None admin] RBAC: Authorizing `identity:get_project(project_id=489e2c3ffdae42c6a83d2b2655912ea9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:06.564179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5a18d-0ede-46c8-9643-f05c845accd8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:06.566021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 208/414] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:06 2022] GET /identity/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:06.571439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:06.682631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd753a3-38d8-498f-9f7c-087c97cecb76 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:06.731041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa98ea4-3bfc-4ed6-9e28-96ee5700dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:06.731860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa98ea4-3bfc-4ed6-9e28-96ee5700dbd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.732192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa98ea4-3bfc-4ed6-9e28-96ee5700dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.732476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa98ea4-3bfc-4ed6-9e28-96ee5700dbd8 None admin] PATH_INFO: `/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9/groups/da4d5556f9e140e488b15d4f056211ff/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.736852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa98ea4-3bfc-4ed6-9e28-96ee5700dbd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=489e2c3ffdae42c6a83d2b2655912ea9, group_id=da4d5556f9e140e488b15d4f056211ff, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:06.738338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa98ea4-3bfc-4ed6-9e28-96ee5700dbd8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:06.759214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/415] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 9 06:50:06 2022] PUT /identity/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9/groups/da4d5556f9e140e488b15d4f056211ff/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 06:50:06.812223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2e570-fd44-40bd-b140-27464b6be0df None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.812573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2e570-fd44-40bd-b140-27464b6be0df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.812783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2e570-fd44-40bd-b140-27464b6be0df None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.814501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 209/416] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:06.821410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:06.821729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:06.821974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:06.847439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:06.848679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:06.862160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:06.946553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/417] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:06.979833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2e570-fd44-40bd-b140-27464b6be0df None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:07.034184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:07.115128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2e570-fd44-40bd-b140-27464b6be0df None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:07.151874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b83d83-b4ae-45ad-a04d-7b87d60943d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:07.152512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b83d83-b4ae-45ad-a04d-7b87d60943d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.152795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b83d83-b4ae-45ad-a04d-7b87d60943d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.153211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b83d83-b4ae-45ad-a04d-7b87d60943d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.165087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b83d83-b4ae-45ad-a04d-7b87d60943d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-13b83d83-b4ae-45ad-a04d-7b87d60943d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:07.169991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:07.170836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:07.172072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 210/418] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:07.177362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:07.252011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6118e1-6e7a-400f-b2ac-bdc644084676 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.252446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6118e1-6e7a-400f-b2ac-bdc644084676 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.252786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6118e1-6e7a-400f-b2ac-bdc644084676 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.254436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 211/419] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:07.259036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.259320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.259612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.296627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:07.297825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:07.312283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7e244-6c55-4f37-bc1d-64533086af89 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:07.383437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 212/420] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:07.393658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:07.394290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.394578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.394856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.395542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:07.395874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:07.397250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:07.414978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/421] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:50:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:07.421233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:07.423249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:07.434876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:07.454218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:07.544152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ccd3-9b86-4474-8cef-10a091ae34c4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:07.568869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1380f0-6e76-442d-b71a-9c823dc12099 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:07.586610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:07.587306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.587510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.587787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.657653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:07.658691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.659180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.659688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.665779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:07.667243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:07.688472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:07.694881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:07.701247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:07.702062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:07.702684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:07.704669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/422] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:07.710163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:07.780857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 213/423] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:07.786981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:07.854847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:07.901398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3057a-b297-4446-a587-82db745b93c2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:07.930484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:07.942093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:07.942723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:07.943000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:07.943265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:07.944716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:07.946134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:07.946388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53191389-f0d2-43a8-9745-9633fb87c651 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:07.947980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 214/424] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:07 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:07.955090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.052876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.053504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.053789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.054064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.054799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:08.055107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.056567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.066962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43243c0b-435f-4504-8f84-f9282560dfeb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.076793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/425] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:08.082195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.107819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.108382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.108652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.108926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.110494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.119626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.119626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 215/426] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:07 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:08.119814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.188874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:08.234382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e58ac-738b-4a06-9ee3-a53a88ae5929 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.251401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157a94-357b-484e-9a6f-3ffe2abae285 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.272377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.273327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.273662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.273939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.274670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=ec2591650a2e438db15b3fccbca1ca57) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:08.275027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.277206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557a97d-e974-4b74-84d8-d67f28427618 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.285449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 216/427] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:08.287089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794b837-3777-4a52-b1b1-b11c54dbda30 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.287377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794b837-3777-4a52-b1b1-b11c54dbda30 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.287717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794b837-3777-4a52-b1b1-b11c54dbda30 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.288050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:08.289012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 217/428] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:08.295299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.295626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.295890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.300674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.302645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.302934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.303213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.332381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:08.333607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:08.461191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.462664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.484519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 218/429] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:08.511920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.577401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:08.582955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/430] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:50:08.588601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.620003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:08.656617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71090bb2-ccdb-4c80-97c5-28bcec01bcd7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.705030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.705675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.705957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.706237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.720377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:08.725408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de590c-a192-4ead-8a18-2912f7a4cac9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:08.727658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:08.728530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:08.729388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 219/431] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:08.734969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.770097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.771008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.771289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.771762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] PATH_INFO: `/v3/projects/ec2591650a2e438db15b3fccbca1ca57` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.773683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] RBAC: Authorizing `identity:get_project(project_id=ec2591650a2e438db15b3fccbca1ca57)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.775909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.777576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/432] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/projects/ec2591650a2e438db15b3fccbca1ca57 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:08.783800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.863630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cbb9f-9666-4a8a-a9af-c0744cf680e3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.884536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:08.898760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:08.907111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16266b7c-b827-4e44-98d9-c41caa79c750 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:08.907417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.908078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.908362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.908637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.909308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:08.909617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.912049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.923637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 220/433] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:08.928346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:08.953980 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:08.954902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:08.955356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:08.955885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:08.957240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:08.958893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:08.961805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/434] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:08.968268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.058592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867a0ad-4813-4b9c-89d0-0321ea78e6bf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.088955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec719a83-57aa-457b-8dfb-2b08badf63d1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.100247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fc467-f263-4523-bcbc-bae5edcb5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.100839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fc467-f263-4523-bcbc-bae5edcb5f7d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.101128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fc467-f263-4523-bcbc-bae5edcb5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.101405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fc467-f263-4523-bcbc-bae5edcb5f7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.134157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.134771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.135048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.135340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.136074 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=ec2591650a2e438db15b3fccbca1ca57) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:09.136381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.138340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d386686-bb48-4d89-8928-8d16ac6d8a1c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.144519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/435] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=ec2591650a2e438db15b3fccbca1ca57&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:09.145722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef400d9-f07d-40e0-9f83-4b26f7636371 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.146004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef400d9-f07d-40e0-9f83-4b26f7636371 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.146328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef400d9-f07d-40e0-9f83-4b26f7636371 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.147545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/436] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:09.154154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.154456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.154705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.165962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.174437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:09.175779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:09.203582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.209275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fc467-f263-4523-bcbc-bae5edcb5f7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.212320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fc467-f263-4523-bcbc-bae5edcb5f7d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.264885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/437] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:09.288442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.339885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 221/438] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:50:09.341470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927f2e9-ad60-4837-938d-7512f3dd76aa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.341760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927f2e9-ad60-4837-938d-7512f3dd76aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.342444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927f2e9-ad60-4837-938d-7512f3dd76aa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.344040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 222/439] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:09.345972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927f2e9-ad60-4837-938d-7512f3dd76aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.410611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8455a9a-b3d8-4b56-9444-871018d3cc9b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.440880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.452161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89702067-2010-4dc9-84d1-e98df8f2dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.452810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89702067-2010-4dc9-84d1-e98df8f2dcac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.453096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89702067-2010-4dc9-84d1-e98df8f2dcac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.453368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89702067-2010-4dc9-84d1-e98df8f2dcac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.468286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89702067-2010-4dc9-84d1-e98df8f2dcac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:09.469759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927f2e9-ad60-4837-938d-7512f3dd76aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-89702067-2010-4dc9-84d1-e98df8f2dcac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:09.473058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:09.473936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:09.474714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:09.476084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/440] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:09.477212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.477496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.477768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.503412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:09.505546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:09.509704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.510351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.510630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.510903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.512093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.513868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.515491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 223/441] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:09.517242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.524102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.581463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/442] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:09.582313 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.629856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed1ff6-8091-4e7d-8821-09e434374dd7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.663371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.663997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.664272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.664545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.665198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:09.665496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.666850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.675452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 224/443] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:09.676103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.686973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.689120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dab7f5-eac4-4ef5-affc-a8847d438bd6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.723013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.723671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.723955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.724227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.725338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.726760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.728340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/444] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:09.729093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.746925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.781732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7593-ca0b-4a92-8689-5cdd6cd73867 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.814098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.814724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.815002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.815274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.825750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:09.828675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c38ed8-e1e2-4f68-a331-261e3df6dfa0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:09.830769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:09.831797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:09.832576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 225/445] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:09.833844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:09.863303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.863943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.864223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.864494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.901014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.932883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.934966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.935557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef30a7-0365-4fd9-9496-ca9e10ca6ced None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:09.951020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:09.968711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:09.969306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:09.969587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:09.969858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:09.970581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=036ddec2546a4fe7bd3b8eca42111ba0, scope.project.id=9670df9796144337814dd085d2d3de3e) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:09.970887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:09.972346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:09.977989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 226/446] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=036ddec2546a4fe7bd3b8eca42111ba0 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:09.978763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.039771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/447] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:10.046036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.092568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029510b3-3de8-4ed6-bf58-70ec43914c7d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.124569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:10.131295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.131944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.132239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.134161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.134161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:10.134161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.135472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.144115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 227/448] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:10.149431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.159676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997208a5-ea31-482b-846e-f878cc4bd22a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.179105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:10.200486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.201231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.201584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.201934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] PATH_INFO: `/v3/projects/9670df9796144337814dd085d2d3de3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.203840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] RBAC: Authorizing `identity:get_project(project_id=9670df9796144337814dd085d2d3de3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.206177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.208749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/449] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/projects/9670df9796144337814dd085d2d3de3e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:10.216122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.268003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ce8fc-b4d7-4706-9785-ff103683817f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.320309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.320918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.321197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.321470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.337145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a1ab3-f057-4fea-9e72-defca009b53a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.373058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.373691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.374061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.374337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] PATH_INFO: `/v3/groups/312a36e1e62642d0b76a62f93979379e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.375448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] RBAC: Authorizing `identity:get_group(group_id=312a36e1e62642d0b76a62f93979379e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.376933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.378787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/450] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/groups/312a36e1e62642d0b76a62f93979379e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:10.384259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:10.384468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.391681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.393139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.436774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:10.483050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a9a6-494b-4904-8a78-adb28307e902 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.491826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 228/451] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:50:10.498016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.517855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.518466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.518755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.519040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.519781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] RBAC: Adding query filter params (group.id=312a36e1e62642d0b76a62f93979379e, role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=9670df9796144337814dd085d2d3de3e) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:10.520091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.521519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.527329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/452] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/role_assignments?group.id=312a36e1e62642d0b76a62f93979379e&scope.project.id=9670df9796144337814dd085d2d3de3e&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:10.596249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:10.603766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9db32d-f51e-4dbc-8fc3-95bcee5c008d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.640377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.641026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.641302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.641570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] PATH_INFO: `/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.642733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] RBAC: Authorizing `identity:get_project(project_id=489e2c3ffdae42c6a83d2b2655912ea9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.644234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.645909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 229/453] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/projects/489e2c3ffdae42c6a83d2b2655912ea9 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:10.651684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.755282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a974991-d2f1-4b06-90ef-ba7be886ff51 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.773235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:10.792239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68eead-db5f-4ea7-81f9-d33bf60a9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.793132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68eead-db5f-4ea7-81f9-d33bf60a9e4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.793457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68eead-db5f-4ea7-81f9-d33bf60a9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.793744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68eead-db5f-4ea7-81f9-d33bf60a9e4c None admin] PATH_INFO: `/v3/groups/da4d5556f9e140e488b15d4f056211ff` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.794872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68eead-db5f-4ea7-81f9-d33bf60a9e4c None admin] RBAC: Authorizing `identity:get_group(group_id=da4d5556f9e140e488b15d4f056211ff)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.796354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68eead-db5f-4ea7-81f9-d33bf60a9e4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.797925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/454] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/groups/da4d5556f9e140e488b15d4f056211ff => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:10.804060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:10.904901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad822cf-1a82-4073-9627-4c42b1c23c65 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:10.936993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:10.937585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:10.937858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:10.938124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:10.938838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] RBAC: Adding query filter params (group.id=da4d5556f9e140e488b15d4f056211ff, role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=489e2c3ffdae42c6a83d2b2655912ea9) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:10.939139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:10.940565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8ea47-dbf3-4826-81b4-2beb69a48e78 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:10.946044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 230/455] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 9 06:50:10 2022] GET /identity/v3/role_assignments?group.id=da4d5556f9e140e488b15d4f056211ff&scope.project.id=489e2c3ffdae42c6a83d2b2655912ea9&role.id=d58e69f2b822466daca66a9696521a24 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:13.331346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329ec50-c241-4d31-99b8-c1e2bf6393e3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.331915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329ec50-c241-4d31-99b8-c1e2bf6393e3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.331915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329ec50-c241-4d31-99b8-c1e2bf6393e3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.333501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/456] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:13.339025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.339425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.339589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.361136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:13.362650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:13.394518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:13.461851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 231/457] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:13.507808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329ec50-c241-4d31-99b8-c1e2bf6393e3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:13.649424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329ec50-c241-4d31-99b8-c1e2bf6393e3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:13.691081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:13.691753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.692039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.692315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.708375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:13.709839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a61ef11e-c347-44d8-929e-296fab4a347f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:13.724707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:13.725627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:13.726484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:13.726484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:13.726484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:13.726484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/458] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:13.736721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:13.749249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a40eb8-89cf-4133-abdb-ee8b657a9ab7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.749529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a40eb8-89cf-4133-abdb-ee8b657a9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.749812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a40eb8-89cf-4133-abdb-ee8b657a9ab7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.751263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/459] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:13.759017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436c84f-ce5d-4557-9ed7-b3f559cebf44 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.759298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436c84f-ce5d-4557-9ed7-b3f559cebf44 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.759580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436c84f-ce5d-4557-9ed7-b3f559cebf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.799897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2436c84f-ce5d-4557-9ed7-b3f559cebf44 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:13.801176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2436c84f-ce5d-4557-9ed7-b3f559cebf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:13.858738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:13.880706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc59f99-e112-432d-a2fe-11ba69b75fd9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:13.904064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/460] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:13.905385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec42527-b3f7-40ad-8090-d9cf5d469eac None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.905695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec42527-b3f7-40ad-8090-d9cf5d469eac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.906039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec42527-b3f7-40ad-8090-d9cf5d469eac None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.911727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/461] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:13.913237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.913779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.914248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.930318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:13.930318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:13.930318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:13.930318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:13.930730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:13.930730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:13.931873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:13.946005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 232/462] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:13.946864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:13.948029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:13.950310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:13.966800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.025542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/463] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:14.026203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.063095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620e02c-8337-4bf2-b892-e759c7120349 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.099006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.099701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.099945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.100222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.111340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.112853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.127041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c2244d40-8813-4086-9885-f85a1fc2b599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:14.127820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:14.128749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:14.129571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:14.129571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:14.129571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:14.129571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 233/464] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:14.130275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7bd1a-76cf-4aa0-aca3-d2cb72b1064c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.130372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7bd1a-76cf-4aa0-aca3-d2cb72b1064c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.130590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7bd1a-76cf-4aa0-aca3-d2cb72b1064c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.131843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 234/465] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:14.132630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7bd1a-76cf-4aa0-aca3-d2cb72b1064c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.141653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91003c78-19b7-48ed-9eec-8466e2967685 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.178517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.179049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.179447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.179768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.180410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.181835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.204062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/466] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:50:14 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:14.205017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.215797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.264322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7bd1a-76cf-4aa0-aca3-d2cb72b1064c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.307157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.307997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.308352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.308722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.321407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.323047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cc682ff4-8e8c-4390-a9e0-4a006a26df43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:14.343380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:14.344827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:14.345679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:14.345679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:14.345679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:14.346291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cc902-ac82-4b3d-8de0-aacec5e6c6a5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.349986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 235/467] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:14.352416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.354575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.355040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.377794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:14.379452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:14.390236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.390954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.391329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.391707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.392484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:14.392857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.394352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.407146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/468] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:14.408112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.426410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.467195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 236/469] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:14.468380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.524738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.553983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d801-7cd1-419c-a22d-2018c9a75726 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.591664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.592426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.592728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.593048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.593768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:14.594142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.595607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.595977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb40f2-0a7a-401f-8568-16c86ebfd131 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.608501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/470] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:14.613809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.641953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.643098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.643967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.644725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.645532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.648120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.681313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 237/471] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:50:14 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:14.682712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.697445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.732181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703a675-ae88-46bd-9912-f93c69016e78 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.781542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.782594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.783042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.783471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.798426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.801540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:14.820557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:14.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:14.822341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:14.822341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:14.822341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:14.826101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/472] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:14.835115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:14.851663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519f3a-7468-4a55-a300-f6c51684e00f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:14.858275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:14.894912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37139e-b4b2-483c-8822-eabd83de5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:14.895533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37139e-b4b2-483c-8822-eabd83de5621 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.895864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37139e-b4b2-483c-8822-eabd83de5621 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.896200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37139e-b4b2-483c-8822-eabd83de5621 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.896848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37139e-b4b2-483c-8822-eabd83de5621 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:14.898297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37139e-b4b2-483c-8822-eabd83de5621 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:14.925605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 238/473] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:50:14 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:14.926858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b35ca-8491-4fe3-b999-60f1efdeb8b5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.927149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b35ca-8491-4fe3-b999-60f1efdeb8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.927422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b35ca-8491-4fe3-b999-60f1efdeb8b5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.929940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 239/474] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:14.936289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:14.936569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:14.936846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:14.957658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:14.958967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:14.975905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cdc09-e267-402b-96e9-222731d9c495 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:15.005261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:15.026176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:15.026870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:15.027159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:15.027578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:15.028578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:15.028578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:15.030077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:15.046557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/475] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:15.051837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:15.057973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 240/476] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:15.085637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:15.217565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216138e-0f86-4268-a6dc-735fc9e9243f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:15.254391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:15.264765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3264a8-13c1-4c5d-9322-c2ca79c3c7af None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:15.275995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:15.276941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:15.277412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:15.279115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:15.280274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:15.281964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:15.306658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:15.307533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:15.308211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:15.308509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:15.309803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:15.312357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:15.315007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/477] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:50:15 2022] POST /identity/v3/users => generated 249 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:50:15.326292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 06:50:15.326292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 06:50:15.326292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [parameters: {'id': 'b74ae4f028ed4df2ae3ddcdb02d7141c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 06:50:15.326292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() Aug 09 06:50:15.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 06:50:15.346791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 06:50:15.348141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 06:50:15.348975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 06:50:15.349849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 06:50:15.350654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': 'b74ae4f028ed4df2ae3ddcdb02d7141c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:15.351625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 06:50:15.352578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:15.354111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 241/478] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 9 06:50:15 2022] POST /identity/v3/roles => generated 144 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:15.355910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:15.408695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:15.475789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde1ccd-020c-487c-841d-91338d6d2137 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:15.528078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037ee5a-e6b4-4d48-91d9-f0f463dcfe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:15.529027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037ee5a-e6b4-4d48-91d9-f0f463dcfe57 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:15.529502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037ee5a-e6b4-4d48-91d9-f0f463dcfe57 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:15.529999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037ee5a-e6b4-4d48-91d9-f0f463dcfe57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:15.545830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037ee5a-e6b4-4d48-91d9-f0f463dcfe57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d037ee5a-e6b4-4d48-91d9-f0f463dcfe57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:15.551467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:15.552352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 06:50:15.553194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:15.555119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/479] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:15.562220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:15.674014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67004795-99eb-426e-aee0-d6e331d82ea9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:15.710904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:15.711851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:15.712331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:15.712775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:15.713621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:15.714096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:15.716104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5579dd6-ab0c-426b-8e9f-f475d7929477 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:15.726797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 242/480] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:16.689371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8f395-3211-4ba0-a5d4-c0c976868f67 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:16.690369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8f395-3211-4ba0-a5d4-c0c976868f67 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:16.690852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8f395-3211-4ba0-a5d4-c0c976868f67 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:16.692430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/481] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:16.697814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:16.698608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:16.699086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:16.721608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:16.723133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:16.741712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:16.808358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 243/482] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:16.842153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8f395-3211-4ba0-a5d4-c0c976868f67 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:16.977022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:16.984883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8f395-3211-4ba0-a5d4-c0c976868f67 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.030943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924e4a9-c038-4a04-8f9b-79a29809215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.031588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924e4a9-c038-4a04-8f9b-79a29809215f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.031894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924e4a9-c038-4a04-8f9b-79a29809215f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.032165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924e4a9-c038-4a04-8f9b-79a29809215f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.049702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924e4a9-c038-4a04-8f9b-79a29809215f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f924e4a9-c038-4a04-8f9b-79a29809215f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:17.054394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:17.055358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:17.056194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/483] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:17.068795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.202746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731d770-9f7f-432f-93e1-f2d4761e20e5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.221733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714590b7-3ac4-4611-907c-757392e9e3d9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.222044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714590b7-3ac4-4611-907c-757392e9e3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.222284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714590b7-3ac4-4611-907c-757392e9e3d9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.223994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/484] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:50:17.229168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b81f7-f953-4d7e-9b01-7af347589993 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.229497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b81f7-f953-4d7e-9b01-7af347589993 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.229756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b81f7-f953-4d7e-9b01-7af347589993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.238311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.238917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.239195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.239469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.240156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:17.240455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:17.241817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:17.251009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 244/485] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:17.256606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.268664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8b7b81f7-f953-4d7e-9b01-7af347589993 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:17.269264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:17.270993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b7b81f7-f953-4d7e-9b01-7af347589993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:17.358488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0bec0-2a82-4a54-868f-a530307c65d0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.389658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/486] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:17.390929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a89647-1541-478b-aa70-2e77652bf237 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.391219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a89647-1541-478b-aa70-2e77652bf237 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.391484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a89647-1541-478b-aa70-2e77652bf237 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.392861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/487] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:17.393163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.393797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.394079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.394352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.397832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.398119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.398391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.405961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:17.407429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:17.417450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:17.418794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:17.422180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:17.424290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:17.426386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:17.426386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:17.426386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:17.426386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 245/488] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:17.431632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.489575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/489] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:17.490355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.496817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:17.539822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3b494-3c91-40f3-8ace-888a1eecc683 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.551987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:17.573943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.575176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.575493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.575799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.576450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:17.576750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:17.578131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:17.595939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 246/490] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:17.596851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.620652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db62b-235b-41d7-8daa-ebf2c4924f4b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.658769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.659469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.659778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.660105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.672789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:17.677986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:17.678914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:17.679749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/491] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:17.680906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.702439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446a7a2-387d-43d0-aab0-0ff77e389d5d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.726966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:17.739937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.740521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.740770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.741040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.751628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:17.756504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:17.757540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:17.758482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:17.762880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 247/492] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:17.763797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.778984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:17.802255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd60a35-add8-4bce-b91f-e7533ce6d894 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.845690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.846574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.847100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.847616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.859485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:17.861849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:17.869051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73d231-99fa-4304-89fd-8cd04ed238ad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:17.878149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:17.879306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:17.880318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:17.882336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/493] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:17.883336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.910104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:17.910716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:17.911209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:17.911480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:17.912555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:17.912863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:17.914503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:17.925241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 248/494] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:17.926053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:17.982783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:17.997783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd451b47-b366-42b6-95e6-799d9f0d665f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.034901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.035937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.036475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.036982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.037842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:18.038345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.038932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce91e71-199f-4185-a622-a7736727f77a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.040446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.049864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/495] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:18.050999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.053747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.075716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.076780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.077061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.077483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.078330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:18.078849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.080975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b02903-64d2-4efa-b931-8fd23a505d24 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.124060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 249/496] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:50:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:18.125342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41af626-85de-496e-9047-ae7fd525b4d8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.125634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41af626-85de-496e-9047-ae7fd525b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.125912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41af626-85de-496e-9047-ae7fd525b4d8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.127485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 250/497] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:18.132254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41af626-85de-496e-9047-ae7fd525b4d8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.161856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caa7d0-9cb9-4359-9395-58a5f050dbb0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.197981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.198634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.198888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.199221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.210042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.211488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:18.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:18.226804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:18.227733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:18.227733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:18.227733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:18.231962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/498] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:18.232844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.242028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41af626-85de-496e-9047-ae7fd525b4d8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.243737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.280441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.280998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.281275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.281571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.291508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.293010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.294905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-db7391ac-16df-4648-811b-ef1b11ac9c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:18.310261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:18.311257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:18.312121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:18.312121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:18.312121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:18.316126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 251/499] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:18.317390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.317676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.317945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.335341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:18.336576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:18.346546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf4871-c2c2-49e4-a9a9-d6e3b8edce2d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.383001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.383610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.383916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.384190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.411023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 252/500] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:18.420629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.463636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.465248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.510704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.526588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.528750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec08b7-0390-47dd-a14c-3c5da22ed6cb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.563932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.564455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.564734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.564999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.565641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:18.565936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.567376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.578274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/501] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:18.583424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 253/502] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:18.583424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.583998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.684322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cab53-c94b-4a69-a99d-cb9815ac4208 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.684599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf62fb-516c-4492-96d4-b1c2494ebd3c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.717862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.718462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.718736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.718828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.719266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.719353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.719644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.719780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:18.719918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.720032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.721283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.729735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:18.732664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/503] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:18.733444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:18.735974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:18.737000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 254/504] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:18.737801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.741146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.755481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.838735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293ae57-e4aa-4d80-a822-ecd5b7ff7846 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.839437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b66b35-284e-4f28-912f-b0c9a4223fe6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:18.873568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.874315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.874591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.874860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.875698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:18.876326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.876610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.876884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.884630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.885231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:18.886064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.886640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7b3fd733-b8c0-4ab5-a591-6a745b1a0e5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:18.899168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:18.900068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:18.900942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 255/505] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:18.901493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:18.908247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:18.909072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:18.909072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:18.909072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:18.909232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17669945-246d-4301-94bf-5c758b820453 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:18.909232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17669945-246d-4301-94bf-5c758b820453 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:18.909232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17669945-246d-4301-94bf-5c758b820453 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:18.909399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/506] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:18.909399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.909523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 256/507] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:18.909523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17669945-246d-4301-94bf-5c758b820453 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:18.969455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:18.974781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.008572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d164b4-8b51-4dda-a042-aadece1fbd76 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.015828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17669945-246d-4301-94bf-5c758b820453 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.042704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.043858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.044135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.044403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.049808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.050413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.050698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.050973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.051661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:19.051761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.051966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.053110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.053328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.061664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 257/508] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:19.066514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b17124e0-cb78-48ac-9d49-1fde4c45b6b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:19.066614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:19.067747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:50:19.068940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:19.071374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/509] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:19.072593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.072872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.073142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.091153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:19.092033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:19.157695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/510] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:19.158527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.172319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080240-6af9-4968-a258-183bebcbb053 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.198007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.207295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.207937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.208219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.208495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.209155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:19.209458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.210818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.216504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.222847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 258/511] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:19.223655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.260332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde6f81-8875-431a-910a-541b687319f4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.294087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.294700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.294981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.295250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.296025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:19.296330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.297745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.322389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b8753-bacc-4736-963e-23eded7d5fa8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.334748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/512] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:19.336922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.358354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.358914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.359192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.359459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.369428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.370882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:19.385582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:19.386505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:19.387570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:19.387570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:19.387570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:19.388083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 259/513] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:19.389026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.427919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.436347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649d77-5934-452c-afa5-9f3e1f3aa077 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.450380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.470686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.471290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.471568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.471874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.472590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:19.472891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.474318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.488839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e47403-1720-43d6-854b-700b72340b1b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.491488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/514] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:19.492319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.524681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.525315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.525594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.525865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.536706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.538150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:19.552198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:19.553057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:19.553863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:19.553863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:19.553863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:19.553863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 260/515] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:19.554110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.593554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93561b5a-38a2-401e-9f7e-fc5af37499b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.627879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.628433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.628713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.628995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.635529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.637476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.644447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:19.649123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:19.649971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:19.650808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/516] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:19.651265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.659208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f3b8f-3fc9-433e-bfb8-58849f798a1a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.683125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.694246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.694858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.695143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.695419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.752683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe67515-a769-4a43-87eb-656468db3969 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.764379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.765850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.786972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.787579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.787888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.788164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.788799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:19.789101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:19.790467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:19.801783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/517] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:19.802565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.854756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.868393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 261/518] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:19.869230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:19.900109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:19.908990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b5be8-f7d5-4618-a46b-d821bfcdde22 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:19.942555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:19.943159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:19.943438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:19.943744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:19.969093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14b8ee-56f5-4720-a346-bd55472e2901 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.002465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.003072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.003351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.003648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.004289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:20.004588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.005947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.011091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.013225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.018489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 262/519] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:20.019295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.067238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.104599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.116479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/520] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:20.117331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.118907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38f17-7f2a-4792-8b80-71a591560d70 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.154256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.154791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.155072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.155344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.156068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:20.156376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.157804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.175425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 263/521] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:20.176298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.227301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef304-104f-4551-8c52-7181cfa905e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.272463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.273113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.273724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.274118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.282349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.283852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.295544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c9d6c-0c88-445a-9c69-9daefa99ab3b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:20.295730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:20.296549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/522] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:20.297416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.308670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.335407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.336077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.336361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.336638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.347350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.348846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.358804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:20.363982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:20.364886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:20.365719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:20.365719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:20.365719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:20.365719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 264/523] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:20.366187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.411839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244439be-c1ea-4b2d-a1ce-71f3efaf431a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.449112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.450433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.450776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.451124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.451869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.453548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.480743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/524] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:50:20 2022] POST /identity/v3/users => generated 247 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:20.481624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.482636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b451c2-2118-4cdc-9deb-659012968c26 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.523076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.523760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.524042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.524315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.537211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.539142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:20.555952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:20.556955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:20.557834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:20.557834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:20.557834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:20.562884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 265/525] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:20.563825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.580940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.609380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46e98-855e-4baa-a333-4a7141e3ae57 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.619230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.649939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.650545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.650823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.651092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.651851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=d80a908c1da74b3fb3e82c8277556e81) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:20.652153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.653655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.659726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/526] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/role_assignments?user.id=d80a908c1da74b3fb3e82c8277556e81&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:20.661138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.696847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddca06-7cd0-4789-8545-e4b0ba80df72 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.743462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.744153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.744431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.744726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.745727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:20.746737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.748585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.773906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 266/527] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:20.779641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.797668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e5fdd-f8f9-4388-9f18-cc0ed950ea75 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.849638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.850259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.850544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.850818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.851466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:20.851889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.853379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.861622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.875228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/528] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:20.876511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:20.907217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a18c3-40e3-4d09-bb0d-7472eb79055c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:20.911857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:20.956246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:20.956780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:20.957056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:20.957350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:20.958862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:20.959163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:20.960547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:20.973600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 267/529] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:20.974726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.006421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c0866-a367-4a58-9c87-c78043c407fe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.051829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.056310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.056700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.057377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.083065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c280-9281-4858-894a-fc318a159c80 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.114054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:21.121128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.122782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.123261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.123791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.131721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.133403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.144309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.145744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:21.146357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:21.147159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:21.148318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:21.150397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 268/530] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:21.151530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.188519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:21.267113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1b924-5416-46f1-939b-4dbfb1967e17 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.274964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/531] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:21.276183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.320737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.321454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.321730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.321997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.332279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.334367 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:21.351449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:21.352485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:21.353328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:21.355743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 269/532] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:21.357053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.394238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a55d0-eef5-456f-94f1-a2309e6448ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.427278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:21.431233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:21.435202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.436225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.436711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.437201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.438102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:21.438611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.440295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.470802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/533] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:21.472085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.489341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259e1f-5ff6-47f6-a1ee-d85ffc4d012f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.530194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.530770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.531049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.531319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.546315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.548470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:21.566850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:21.568021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:21.569135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:21.569135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:21.569135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:21.571728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 270/534] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:21.573545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.596451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b8432-eced-4645-979c-e2e0a28a5d1a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.639481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.640473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.640890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.641301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.642153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:21.642605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.644307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.663186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/535] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:21.664544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.703071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c151d0-df43-4971-af75-df28073b34ff None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.705977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:21.727031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:21.742326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.743275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.743737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.744605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.745453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=337f5152bd51454cb58d6b2092d5cf8b) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:21.746144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.748608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.763632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 271/536] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/role_assignments?user.id=337f5152bd51454cb58d6b2092d5cf8b&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:21.763632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.818360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd3018-144c-423d-a066-6172c8a93e42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.928664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2ab0-b892-4a4f-9e8a-0bfdbbbc449a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:21.937999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.938625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.938904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.939174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.941225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:21.941540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.943165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.956114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/537] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:21.962308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:21.976918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:21.978100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:21.979034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:21.979818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:21.981393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=6c7d784c96b044ef8b383767e8c1cb08) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:21.982027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:21.984159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:21.991046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 272/538] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/role_assignments?user.id=6c7d784c96b044ef8b383767e8c1cb08&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:22.116195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4746f3-83e8-4bd1-a297-571578a23286 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:22.132032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:22.164658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:22.165318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.165601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.165879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.179747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:22.180744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:22.197918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:22.198886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:22.199793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:22.200667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/539] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:22.205852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:22.253145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:22.348939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7ab8-57a0-48db-a958-2a4ba5a09eb2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:22.397861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:22.398481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.398764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.399041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.399764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:22.400077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:22.401528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac44e57-4d48-4688-8d03-b7a4f3f3817a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:22.419391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 273/540] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:22.425429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:22.555447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cf119-33a1-4f7a-bff2-8a65b7f8e4a5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:22.605308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:22.606181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.606470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.606747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.607486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=476a1946148e4758b071e9b6ce788bcd) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:22.607820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:22.609425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9322b5a-4d5a-4f0b-b475-a0f0c3ecc0c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:22.617385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/541] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:22 2022] GET /identity/v3/role_assignments?user.id=476a1946148e4758b071e9b6ce788bcd&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:22.657905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c2965-14dd-40d5-b9e5-7febb5bd571d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.658183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c2965-14dd-40d5-b9e5-7febb5bd571d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.658387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c2965-14dd-40d5-b9e5-7febb5bd571d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.659999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 274/542] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:22.665355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7f36-818f-42c7-8dd0-53e01b55f4b2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.665653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7f36-818f-42c7-8dd0-53e01b55f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.665937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7f36-818f-42c7-8dd0-53e01b55f4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.685882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-83ad7f36-818f-42c7-8dd0-53e01b55f4b2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:22.687096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-83ad7f36-818f-42c7-8dd0-53e01b55f4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:22.719023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:22.769337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/543] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:22.794065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c2965-14dd-40d5-b9e5-7febb5bd571d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:22.911325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:22.922541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c2965-14dd-40d5-b9e5-7febb5bd571d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:22.935644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5db33-4e9d-4ef9-9ba5-9a846b86264f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.935849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5db33-4e9d-4ef9-9ba5-9a846b86264f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.936044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5db33-4e9d-4ef9-9ba5-9a846b86264f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.937330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/544] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:22.942875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.943433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.943433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:22.977264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:22.978455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:22.992237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:22.992865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:22.993139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:22.993409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:23.014247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:23.019183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:23.020114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:23.020886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:23.022895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 275/545] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:23.029109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.099822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/546] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:23.139668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.216754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e8282-8461-45df-b4ac-4335f76ca078 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:23.263033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:23.263670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:23.263955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:23.264231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:23.266695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:23.267005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:23.268420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:23.289803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 276/547] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:23.295299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.337067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:23.338376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:23.347010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c256-0445-489f-9e12-d201d254ed0d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:23.419945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:23.420592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:23.420876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:23.421151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:23.447016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:23.452940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:23.453841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:23.454651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/548] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:23.460366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.489119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8efad4-2b82-4839-b8c2-446a27e606cf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:23.536053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:23.536703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:23.537033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:23.537311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:23.558848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:23.560547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:23.587641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:23.588629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:23.590727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:23.590727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:23.590727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:23.595771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 277/549] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:23.596809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.679910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35bd6-5413-4cf7-b328-a8ecf20ac202 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:23.737394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:23.738007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:23.738290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:23.738566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:23.739289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:23.739540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:23.740932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:23.751968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:23.766437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/550] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:23.768220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.811954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:23.821842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718c52f-01ce-4943-81b3-e4dd89f9492c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:23.902300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:23.903463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:23.903773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:23.904049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:23.904700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:23.905003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:23.909221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:23.938930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 278/551] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:23.949147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:23.991482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cf0f0-62c1-4de7-91ab-cc4a4fb04ce2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.075376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8717c4-1981-4107-b5e6-a86ae7ee0948 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.100314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.100963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.101241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.101513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.108179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.113474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.114116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.114407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.114684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.119696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.121976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.123665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.125161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-95183c47-5292-4a99-a1cc-66eb1d75020d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:24.141471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:24.143578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:24.145884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 279/552] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:24.146672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:24.156827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:24.158925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:24.158925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:24.158925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:24.158925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/553] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:24.158925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.159661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0659c69-ee66-4a4e-a07c-842896feb0dd None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.159661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0659c69-ee66-4a4e-a07c-842896feb0dd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.159661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0659c69-ee66-4a4e-a07c-842896feb0dd None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.159661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 280/554] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:24.159661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0659c69-ee66-4a4e-a07c-842896feb0dd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.238275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.275805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0659c69-ee66-4a4e-a07c-842896feb0dd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.281217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7b72-02d0-47ec-9bcd-5777d70a421e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.324340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.325035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.325322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.325600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.326302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:24.326613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.327672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.328136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd67f-66c1-4681-97ff-56a0af9bc189 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.328728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.329222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.329930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.331464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:24.331999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.333737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.347332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 281/555] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:24.348702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.348989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.349263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.362247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/556] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:24.362247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.374486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:24.375750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:24.421433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.455752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 282/557] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:24.456737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.497582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dec6a7-9567-47e1-9a77-e9c4cadeade7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.529020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.538328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.538971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.539253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.539534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.547469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.549021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:24.562486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:24.564014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:24.564956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:24.567781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/558] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:24.568721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.600586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d61d-eae5-40fc-a18c-2747b97b303c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.637486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.638243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.638515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.638791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.673983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaa5fc-6ea0-4ee7-9245-8006bceb48eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.686970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.710128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86876332-9345-424d-86cc-f1abe332a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.710767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86876332-9345-424d-86cc-f1abe332a477 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.711052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86876332-9345-424d-86cc-f1abe332a477 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.711325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86876332-9345-424d-86cc-f1abe332a477 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.712171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.713856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.722094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86876332-9345-424d-86cc-f1abe332a477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-86876332-9345-424d-86cc-f1abe332a477 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:24.727187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:24.728387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:24.729227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/559] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:24.729911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1b633-bac5-4159-8716-565a93563794 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.731129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1b633-bac5-4159-8716-565a93563794 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.731129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1b633-bac5-4159-8716-565a93563794 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.736323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/560] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:24.737151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1b633-bac5-4159-8716-565a93563794 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.791566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.826304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 283/561] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:24.828500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:24.851129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1b633-bac5-4159-8716-565a93563794 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.886638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.888123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.888123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.888377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.889099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:24.889400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.890832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec43487-7009-4a38-aef8-db16f3c09dd9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.908404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/562] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:24.909620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8648c-c70b-4cf0-aef2-ef25936e6890 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.909916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8648c-c70b-4cf0-aef2-ef25936e6890 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.910190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8648c-c70b-4cf0-aef2-ef25936e6890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.919696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:24.928455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0ae8648c-c70b-4cf0-aef2-ef25936e6890 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:24.929664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ae8648c-c70b-4cf0-aef2-ef25936e6890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:24.943172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc70f-f9ab-4f34-9572-91d2a117699c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:24.976775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:24.977379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:24.977658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:24.977932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:24.978590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:24.978890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:24.980343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:24.988708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 284/563] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:24.989484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.001577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/564] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:25.003080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ebceb-e030-433b-927c-9d5eeb6e7202 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.003371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ebceb-e030-433b-927c-9d5eeb6e7202 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.003694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ebceb-e030-433b-927c-9d5eeb6e7202 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.004923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/565] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:25.005617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ebceb-e030-433b-927c-9d5eeb6e7202 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.052663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.102036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2fb85-28df-42c7-91ab-5190045e78d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.119678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ebceb-e030-433b-927c-9d5eeb6e7202 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.135797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.136457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.136785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.137214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.144121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.148813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.150249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.152251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.153373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.153644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.153906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.163537 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:25.164346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.165380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:25.166244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:25.166244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:25.166244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:25.166412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.166614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 285/566] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.166614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-deaa18f1-7782-43b9-add8-7b16bdff5aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:25.184921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.185929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:25.186817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:25.186817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:25.186817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:25.186817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/567] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.187893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.187893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.187893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.206308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:25.207550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:25.268835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c4a11-0a03-4fc2-90a8-4c721641e902 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.274162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/568] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:25.274983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.297988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.307627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.308214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.308500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.308801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.319185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.320652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:25.334372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.335320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:25.336431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:25.336431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:25.336431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:25.336431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 286/569] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.336431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.380620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.381479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969b4c7-0b67-4333-bef4-fad1f8b8c781 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.415424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.416088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.416369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.416643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.426536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:25.431144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.432232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/570] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.433240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.441310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3689171-8720-49de-8035-2ab085adbf86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.475446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.476072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.476349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.476619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.477260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:25.477561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.478913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.491152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 287/571] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:25.491771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.515633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.537170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d0fa5-f47b-4420-a67b-76ce03ae6760 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.572187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.572719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.572998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.573267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.573897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:25.574265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.575770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.586886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/572] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:25.587693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.595249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4331c5-3f87-450d-85e8-fd0b1a91cb86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.596849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.631350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.632002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.632282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.632554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.643069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:25.647705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.653884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 288/573] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.654984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.688039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14868f-0ad0-4e15-b9a1-747f46a25124 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.721079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.721689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.721993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.722265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.722898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:25.723199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.724594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.732586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.737788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/574] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:25.738654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.757323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceb9c7-663d-47c7-a4d3-f837d26635ca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.790538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.791532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.791854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.792125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.792778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:25.793587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.795104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.802968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 289/575] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:25.803784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.831084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.841666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637949f1-4cf4-4385-a5f6-7032e92e2c66 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.876145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.876773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.877052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.877585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.884040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.885477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:25.897183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.898092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:25.898994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/576] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.899504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:25.912780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d7ece-fb4e-4c15-93c3-a4a075f284ea None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:25.945970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:25.946590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:25.946866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:25.947131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:25.954044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:25.955916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:25.960928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:25.967578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:25.968695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:25.969591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 290/577] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:25.970091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.005056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63766d8f-6c54-4b51-8fae-4982987a328f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.037814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.038409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.038684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.038954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.039613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.039919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.041260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.049445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/578] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.050215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.063478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.079209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c21c06-2571-4528-9a61-efb9456e42e1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.113039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.113701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.114083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.114459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.121912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.123429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:26.134561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:26.135680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:26.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 291/579] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:26.140576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.148162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10286a49-8322-4cac-b4bb-b059460a147e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.168091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.183470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.184672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.184953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.185216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.195270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.196728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:26.210865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:26.212053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:26.213119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:26.213119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:26.213119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:26.213119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/580] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:26.213119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.242669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97360-6a96-4175-ae85-83de7422e23e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.276072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.276670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.276951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.277223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.277979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.278281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.279728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.291466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 292/581] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.292397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.297011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.314890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cfe7f-06a5-488e-af65-d82acac80df8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.347914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.348515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.348794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.349061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.349737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.350035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.351433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.363316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/582] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.364064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.397769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a5a9-7cd3-496c-a721-7263ad02c0b4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.397937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.432163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.432785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.433066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.433338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.443584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.445033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:26.459121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:26.464837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:26.465950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:26.465950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:26.465950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:26.465950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 293/583] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:26.465950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.466404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ccf8-ebec-4ac3-a54b-b8bf790cb623 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.500545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.501144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.501421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.501690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.503081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.503368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.505168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.518844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.523083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/584] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.523914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.569260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202c22d-723f-4c86-bec6-31635f4e4618 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.602298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.602899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.603177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.603450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.604116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.604416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.606343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.613905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.618295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 294/585] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.619069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.629172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3d0fe-6a48-4306-b5a2-5f5baf97797a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.662765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.663368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.663700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.663981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.664703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=8f3a4e99f3044d35aa2e310478f3653b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.665009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.666432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.672452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/586] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/role_assignments?user.id=8f3a4e99f3044d35aa2e310478f3653b&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.673236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.723415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6f280-5ea8-4df3-8b49-be02e9ad236e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.743110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.764078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.764918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.765333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.765708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/d80a908c1da74b3fb3e82c8277556e81/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.787026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=d80a908c1da74b3fb3e82c8277556e81, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.788636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.801606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dce9d-19c8-4898-823b-486a08ce95ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.815638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 295/587] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:26 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/d80a908c1da74b3fb3e82c8277556e81/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:26.815638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.883429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.884195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.884794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.885142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:26.886334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:26.887615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:26.889431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:26.913235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/588] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:26.914533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:26.942887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67621d49-38f0-4d9e-a814-e1daabe852d8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:26.947213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:26.987720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:26.988632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:26.989058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:26.989472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.006084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.007975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:27.025407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:27.026377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:27.027277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:27.027277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:27.027277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:27.030183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 296/589] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:27.031219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.060539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.062274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c519-8243-4b0a-b8e0-95750394ea37 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.105778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.106825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.107181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.107714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.117650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.119383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:27.135221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:27.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:27.138076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/590] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:27.139113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.167905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de98743-1538-47d7-8f8e-cfdda89ff0dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.215725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.216902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.217202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.217603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.228889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.230373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.234364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:27.247020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:27.248207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 06:50:27.249240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:27.252085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 297/591] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:27.253275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.268634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7b66-a37f-435e-9e18-9c942a348351 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.316174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.317576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.318284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.318707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.319772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:27.320641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.322705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.344689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/592] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:27.344689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.349314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.381821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a9d6-2b63-47ca-9c36-91f13d5f38b7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.419901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.420670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.421023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.421484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.422292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:27.422667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.424214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.443545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 298/593] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:27.444710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.476826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64faf76-0850-49e6-8685-2597d4bac7f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.517624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.518290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.518575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.518853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.519790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:27.520102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.521807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.523592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.544283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/594] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:27.549460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.570771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8b72-82c8-458b-83ce-4cc387aa7726 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.616396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.617580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.618291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.618724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.622435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.632420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.634496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:27.649848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:27.650913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:27.652046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:27.659238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 299/595] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:27.660580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.666114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904adcbb-b1c9-4a01-9100-de57e1b1fc93 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.703193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.704013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.704442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.705517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.721105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.722780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:27.739046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:27.740220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:27.741336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:27.741336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:27.741336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:27.747225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/596] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:27.748223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.789409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c7038-e044-4ae0-b5d7-1e612d50f253 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.805173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.831643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.832460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.833314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.833314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.844843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.846819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:27.863692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:27.864913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:27.865993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:27.865993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:27.865993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:27.872788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 300/597] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:27.873861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.880392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee24b91-f445-4e83-8868-39c4a743a5fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:27.926246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:27.927160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:27.928073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:27.928519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:27.929702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:27.930153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:27.930954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:27.933142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:27.953565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/598] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:27.953565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:27.998209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58bb7e-3101-4677-b0c9-a811e67b5c7b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.035167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.036171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.036652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.037141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.037991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:28.038488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.040092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.052771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 301/599] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:28.054047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.062243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6cf44-5a15-45ae-8c26-59e34c4181aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.064691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:28.100728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.101566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.101693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.101867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.102502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:28.102801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.104238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.116260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/600] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:28.117087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.164513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3478af-864f-415c-bfff-7c0c4340e0d8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.180977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:28.217934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.218655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.218940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.219213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/337f5152bd51454cb58d6b2092d5cf8b/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.239494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=337f5152bd51454cb58d6b2092d5cf8b, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.240004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d1b0b-f882-4c08-9f63-a7af7c720ff7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.241057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.266193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 302/601] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:28 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/337f5152bd51454cb58d6b2092d5cf8b/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:28.266193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.298287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.299357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.299892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.300389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.316823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.318728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:28.337244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:28.338192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:28.339046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:28.348178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/602] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:28.353324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.384167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b449bc-48da-4f5b-86bb-456f15516410 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.404491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:28.434436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.435331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.435848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.436265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.446237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.448596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:28.463239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:28.464427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:28.465659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:28.468426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 303/603] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:28.479792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.525427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:28.582777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50b85d-7308-4ee6-afb0-e7f2d5e7203f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.618580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656522b-4e82-4ea3-8749-04a477b3cf2b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.627954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.628789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.629221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.630032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.630930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:28.631797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.634195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.650252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/604] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:28.656350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.660191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.661673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.661673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.661673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.662366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:28.662742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.664324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.678252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 304/605] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:28.685206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:28.775522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:28.790419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:28.810534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b0b43-92e7-4817-b86f-009fd76148ff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.812563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d960-da61-411e-9df9-4431cb6f81fc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:28.847191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef827-6edc-435c-8f00-15ef70e146b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.847894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef827-6edc-435c-8f00-15ef70e146b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.848177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef827-6edc-435c-8f00-15ef70e146b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.848491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef827-6edc-435c-8f00-15ef70e146b9 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/6c7d784c96b044ef8b383767e8c1cb08/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.862451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b991e11-399f-4019-bc05-1bbab8fef08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:28.863683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b991e11-399f-4019-bc05-1bbab8fef08e None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.864610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b991e11-399f-4019-bc05-1bbab8fef08e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.865105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b991e11-399f-4019-bc05-1bbab8fef08e None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/476a1946148e4758b071e9b6ce788bcd/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.867332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef827-6edc-435c-8f00-15ef70e146b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=6c7d784c96b044ef8b383767e8c1cb08, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.868809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef827-6edc-435c-8f00-15ef70e146b9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.890308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/606] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:28 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/6c7d784c96b044ef8b383767e8c1cb08/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:28.890308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139076b-fc46-4db7-8b6f-03fd7e2f4964 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.890308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139076b-fc46-4db7-8b6f-03fd7e2f4964 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.890308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139076b-fc46-4db7-8b6f-03fd7e2f4964 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.890308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/607] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:28.891573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b991e11-399f-4019-bc05-1bbab8fef08e None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=476a1946148e4758b071e9b6ce788bcd, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:28.893710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b991e11-399f-4019-bc05-1bbab8fef08e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:28.894964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:28.895255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:28.895583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:28.922077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 305/608] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:28 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/476a1946148e4758b071e9b6ce788bcd/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:28.930742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:28.932055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:29.000951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19779b0a-8521-4ca8-a463-736b15de0565 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.001139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19779b0a-8521-4ca8-a463-736b15de0565 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.001405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19779b0a-8521-4ca8-a463-736b15de0565 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.003414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 306/609] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:50:29.009119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.009413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.009662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.016477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/610] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:29.029868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:29.031118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:29.036841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.124235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:29.151211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265364-477b-46c2-8c5d-2408fce6a811 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.189282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 307/611] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:29.202241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.202888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.203178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.203455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.214550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:29.215315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:29.219768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:29.220698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:29.221921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/612] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:29.226945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.300461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:29.347056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f6bde-5c7a-4d74-ba60-a837b249e4ec None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.350855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bb8ac-e69c-472e-8f7c-fecb97cd346b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.384037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:29.384345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.385269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.385269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.385550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.397734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:29.399309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.400205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.400685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.401152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.402032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:29.403108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:29.403453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:29.404353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:29.405774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 308/613] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:29.408210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:29.410381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.419441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/614] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:29.431627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.576882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a012fc-7ba7-447a-b7f3-d0d0359e8a52 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.608605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:29.616797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988514b-3b28-4b8c-8f99-066ac33c164f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.652368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.653075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.653363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.653635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.654318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:29.654624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:29.656040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:29.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.665297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.665726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.666161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.666929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 309/615] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:29.672848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.678661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:29.681128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:29.698269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:29.699894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:29.700771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:29.700771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:29.700771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:29.702899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/616] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:29.709037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.766722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:29.886709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b41d1c-5ecf-486a-80f8-1655d6cbb3bc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.897522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c91cb-9a39-4a59-af9f-d83f72b79758 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:29.925422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.926061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.926340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.926615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.938549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:29.940028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:29.956158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:29.957097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:29.957978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:29.957978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:29.957978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:29.957978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 310/617] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:29.963693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:29.977738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:29.978355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:29.978634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:29.978907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:29.979549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:29.979880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:29.981981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.006781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/618] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:30.014430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.029103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:30.118139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562221e-cb84-410d-acac-3b8d04292757 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:30.184158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:30.184916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:30.185194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:30.185464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:30.186104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:30.186404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:30.188172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.190771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:30.196593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a492691-f7a5-499f-94e3-ec6b682c9ad7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:30.223292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 311/619] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:30.231641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.259691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:30.260326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:30.260599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:30.260865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:30.273810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:30.275263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:30.296736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:30.297857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:30.298802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/620] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:30.307887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.369895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:30.390002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ad362-6522-40a2-a5b6-3f6c6432a894 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:30.456617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:30.457259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:30.457536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:30.457810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:30.461665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e20e2c-0b26-4c27-a4d7-5987314c0558 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:30.473102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:30.474559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:30.492933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:30.493910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:30.494821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:30.499619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 312/621] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:30.502621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.526074 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:30.526684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:30.527018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:30.527375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:30.528224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:30.528529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:30.530297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.548190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/622] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:30.553226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.599949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:30.706523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08cd2-1406-400e-8c99-7572e5b6da88 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:30.741757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc4fb-3df3-417c-9225-3e27f272e58f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:30.743276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:30.744275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:30.744625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:30.744967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:30.757890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:30.759425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:30.776294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:30.777415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:30.778443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:30.778443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:30.778443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:30.778443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/623] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:30.783060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.821705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:30.822322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:30.822601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:30.825625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:30.826390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:30.826702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:30.828444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:30.867588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 313/624] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:30.874025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:30.893902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:30.919345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:30.968293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd905e-a952-4b66-9f68-d8f35769763a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.014382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.015473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.016038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.016317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.017249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:31.017957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.019585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068702cb-de23-4403-9c4f-dac9d1a5f4ca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.023708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f776f-1b4c-4fb7-8c38-b582ecc79fcf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.035649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/625] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:31.037268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96edba-ddf0-4438-bd18-3fff53062e06 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.037552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96edba-ddf0-4438-bd18-3fff53062e06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.038077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96edba-ddf0-4438-bd18-3fff53062e06 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.040256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/626] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:31.041149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96edba-ddf0-4438-bd18-3fff53062e06 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.062271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.062886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.063162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.063429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.144943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.147017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.160463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.165067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96edba-ddf0-4438-bd18-3fff53062e06 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.197457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.207534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.208430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.208856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.209272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.217239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.219037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e91efa2d-3968-4b92-9a65-dfefdc93e77c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:31.231157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:31.232126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:31.233005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:31.239866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/627] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:31.241533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.241994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.242443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.258614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 314/628] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:31.259640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.264012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:31.265446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:31.339322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/629] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:31.340689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.365445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8c06b-f658-4ca1-8961-546662dd1442 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.380004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.403783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.404398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.404684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.404958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.405656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:31.405960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.407434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.419589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 315/630] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:31.420396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.462096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301b0b-4792-492c-8df2-c8e35587ad69 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.479129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.497466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.498103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.498386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.498663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.508756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.510225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.519590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0461-80ad-4071-b24a-c3c690b16e09 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-07d0bf9b-ae56-4adc-a544-b9fd1df9e613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:31.525568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:31.526479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:31.528116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:31.528116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:31.528116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:31.528116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/631] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:31.529143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9f16-ffe3-4229-a9d8-f44a832e4648 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.529143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9f16-ffe3-4229-a9d8-f44a832e4648 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.529143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9f16-ffe3-4229-a9d8-f44a832e4648 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.530319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/632] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:31.532288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba038a0-c67b-4a1e-9093-5535b9ea7791 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.532288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba038a0-c67b-4a1e-9093-5535b9ea7791 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.532288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba038a0-c67b-4a1e-9093-5535b9ea7791 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.532991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/633] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:31.534335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba038a0-c67b-4a1e-9093-5535b9ea7791 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.557035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.557669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.557950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.558222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.568397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:31.573478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:31.574943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:31.576041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 316/634] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:31.576747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.605967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.640738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba038a0-c67b-4a1e-9093-5535b9ea7791 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.673392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383eea08-1f4f-48b1-a31c-03927fbd1439 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.673885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0c962-7dd3-4ca2-834b-74ee71e49d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.674543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c962-7dd3-4ca2-834b-74ee71e49d62 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.674827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c962-7dd3-4ca2-834b-74ee71e49d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.675107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c962-7dd3-4ca2-834b-74ee71e49d62 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/8f3a4e99f3044d35aa2e310478f3653b/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.689553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0c962-7dd3-4ca2-834b-74ee71e49d62 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=8f3a4e99f3044d35aa2e310478f3653b, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.690971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0c962-7dd3-4ca2-834b-74ee71e49d62 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.703703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/635] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:31 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/8f3a4e99f3044d35aa2e310478f3653b/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:31.704879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.705164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.705437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.716968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.717672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.717879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.718269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.718874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:31.719295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.720964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a4d3c-b666-4af1-b88a-542ee3c615f5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.725973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:31.727549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:31.736101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 317/636] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:31.737411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.737742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.738208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.750330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.762732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:31.764127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:31.823625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/637] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:31.823625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.844381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 318/638] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:31.845227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:31.878281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:31.950201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a714-8b95-485a-a7f6-7fd4d49b563c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.970997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3498cc-5bf3-4513-b0bc-b850c76423e2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:31.986300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:31.986841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:31.987118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:31.987387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:31.988067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:31.988365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:31.989725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:31.999999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/639] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:32.001027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.015004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.015821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.016100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.016364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.018221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.025331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.026964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:32.040410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:32.041405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:32.042272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:32.046337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 319/640] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:32.047193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.111090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803967e-0b8c-4aa4-9ca0-6f6a29500220 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.132118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.150365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.151133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.151409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.151714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.155919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22821a74-a3ef-418c-ace7-bd675377d8c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.162488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:32.167623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:32.168789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/641] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:32.174781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.201782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.202488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.202822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.203096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.214851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:32.219467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:32.220713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 320/642] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:32.221928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.252958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.286072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e25b8-dd9a-4e45-b6fa-234a96338d5c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.321391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.322022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.322308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.322582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.332412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068fab1-71f1-45de-a3d2-7384f10ee3e5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.333915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.335374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:32.349879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:32.350996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:32.352171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:32.352171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:32.352171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:32.352171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/643] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:32.352171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.376077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.376690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.376974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.377504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.378201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:32.378509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.379974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.389338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.392812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 321/644] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:32.393747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.463730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7360af-497b-4762-ba93-25c707639bbb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.488496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.501280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60230de-29a9-48b2-8400-20a7c81c44ff None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.501424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.502038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.502577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.502856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.503582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:32.503852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.505205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.518154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/645] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:32.518154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.543507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.544208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.544493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.544774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.545462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:32.545743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.547106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.554986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 322/646] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:32.555794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.616143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.633717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9929a4-6969-46f8-90db-c1b40b430112 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.658401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bed8c-4e74-46aa-bab6-c7c59e424779 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.669584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.670187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.670466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.670736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.671365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:32.671692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.673044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.685618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/647] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:32.686224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.699136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.699896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.700186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.700464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.701198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=d80a908c1da74b3fb3e82c8277556e81) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:32.701510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.702953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.719644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 323/648] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/role_assignments?user.id=d80a908c1da74b3fb3e82c8277556e81&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:32.719644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.748796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.813740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c66014-08f3-4178-88ea-e15d218cd6cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.849227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dedae2-6efe-44c7-8e57-d3757d09ca28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:32.853103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.853762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.854044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.854369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.874644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.876232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.884215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:32.897195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:32.897833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:32.898110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:32.898378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:32.899275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:32.900195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:32.901068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:32.901068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:32.901068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:32.901617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/649] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:32.904898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:32.911839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:32.913311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:32.932152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:32.933160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:32.934593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:32.934593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:32.934593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:32.934593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 324/650] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:32.936373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.021225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.026657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b4d68-67ee-458f-982c-3ab7ae1f1f01 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.062985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777532d0-065a-420d-81e9-b13d8947695f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.071806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.072977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.073421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.074674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.084677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.086535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.098394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.099013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.099298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.099574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.100254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:33.100557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:33.101556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:33.102627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 06:50:33.103659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:33.104191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.105015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/651] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:33.110586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.121796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 325/652] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:33.122766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.171258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.236100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e80900-f6ff-433d-8486-6e265fdf5cc2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.236938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddaa68-5f2d-4ea0-8633-0a0d4174009c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.273936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.274624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.274904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.275183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.276158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:33.276688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.279029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.281729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.283074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.283529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.284019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.284934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:33.285420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.286994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.293871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/653] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:33.294796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.316137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 326/654] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:33.316137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.325399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.413788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295355-8cc8-4993-be37-9feee0e5a2d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.444693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.453163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146818b0-6a65-44ae-9cfb-00b8c7836d75 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.454733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.455591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.456004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.457045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.464447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.466690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:33.479925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:33.481757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:50:33.482683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:33.487304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/655] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:33.488426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.497803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.499202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.499728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.500176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.511040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.513131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:33.531099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:33.532114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 06:50:33.533028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:33.536161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 327/656] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:33.537118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.579966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.611397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31042298-3397-4027-882e-69ce49bd9e59 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.655248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22b9a-e7ef-4c69-8a44-e44c107f3dd4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.657199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.658864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.659371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.659883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.691697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.692308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.692593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.692868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.693651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:33.693959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.695450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.720194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 328/657] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:33.721037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.738600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.748685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.750580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.829689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08998d-a99f-4c1e-aa62-752bc3365405 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.842629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:33.867632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:33.868255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.868535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.868808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.874571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:33.874732 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/658] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:33.874837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:33.874932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.875023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c6dd2-c834-4109-af56-e5c1335f98d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:33.897902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 329/659] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:33.897902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb842b3-637c-47c4-b949-0c557b83ea96 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:33.897902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb842b3-637c-47c4-b949-0c557b83ea96 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:33.897902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb842b3-637c-47c4-b949-0c557b83ea96 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:33.897902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 330/660] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:33.897902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb842b3-637c-47c4-b949-0c557b83ea96 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:33.982027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710300ac-f3d4-438e-abd8-c5713aa76fd9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:33.987759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.022445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1029b-058f-4398-8595-3fb0647136b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.022878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb842b3-637c-47c4-b949-0c557b83ea96 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.023057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1029b-058f-4398-8595-3fb0647136b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.023340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1029b-058f-4398-8595-3fb0647136b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.023635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1029b-058f-4398-8595-3fb0647136b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.060867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.061503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.061776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.062045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.072544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.074293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.077581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:34.090192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:34.091171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:34.092083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:34.092083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:34.092083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:34.092083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 331/661] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:34.092934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.104827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1029b-058f-4398-8595-3fb0647136b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.107430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1029b-058f-4398-8595-3fb0647136b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.198181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc225f-6a9a-4d70-bd4a-6e68bd0bda25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.218161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/662] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:34.219418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.219725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.219997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.223145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.236433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.237040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.237322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.237599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.239191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:34.240404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:34.309626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.310581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/663] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:34.311337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.316370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.321876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.415243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7553d9e-bec6-41f7-b455-806e4a118aa8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.435286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 332/664] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:34.436899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.450545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.451159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.451444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.451754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.452399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:34.452707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.454113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.466911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/665] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:34.467565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.473222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.541821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.552464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731d5e-169a-47b4-b099-9ff0cd9d3153 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.579983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e79cf3-1044-4699-adb8-55708a99ce10 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.588591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.589231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.589612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.590067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.600879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.602315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.616922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:34.617664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:34.618756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:34.619684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:34.619684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:34.619684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:34.619684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 333/666] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:34.619976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.620131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.620571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.621406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.634647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:34.640039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:34.640968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:34.641781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/667] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:34.642476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.709579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.734114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d045c90-3f37-4962-ab82-50cd4f9d9119 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.749646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3eb62-33ae-424a-8653-671d9baf23c8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.769555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.770485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.770760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.771030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.780339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.782330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.783927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.788583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.789211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.789486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.789753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.797111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.798585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:34.799618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:34.800495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:34.801363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:34.801363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:34.801363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:34.801363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 334/668] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:34.801590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:34.815690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:34.816587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:34.817471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/669] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:34.817958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.920399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e74cf-901a-40b6-bc99-5e272a4fa434 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.929674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8388c-bb73-4d43-bb5c-06785ec0f466 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:34.945506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:34.953537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.954144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.954428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.954701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.955349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:34.955675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.957040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.963003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:34.963545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:34.963887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:34.964156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:34.965320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:34.965617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:34.966944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:34.969139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 335/670] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:34.969935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:34.977230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/671] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:34.978499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.028714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.067501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d376c6c-a7e1-4486-8b1e-7b76eaa9ee48 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.078682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f490316-3374-455c-8ebd-eb5e1ef81acc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.101487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.102090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.102370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.102646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.103284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.103584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.104967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11541934-70e1-43c9-bd61-d5777b148b3a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.112418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.113024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.113334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.113617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.114301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.114609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.116058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.116591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 336/672] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.117966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31518f-0ed4-4066-acfa-c24f805edbda None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.118258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31518f-0ed4-4066-acfa-c24f805edbda None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.118940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31518f-0ed4-4066-acfa-c24f805edbda None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.123209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 337/673] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:35.124183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31518f-0ed4-4066-acfa-c24f805edbda None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.129163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/674] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.129969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.176034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.227694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0252a-eea2-47f3-bf81-adfb068dbc40 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.229033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31518f-0ed4-4066-acfa-c24f805edbda None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.239630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.264221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.264845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.265118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.265386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.265545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.266163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.266441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.266709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.271826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.273251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.276431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.278417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:35.284784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:35.285678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 338/675] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:35.287103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f5a993bf-dd57-4a14-b559-bd7368f6616f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:35.297300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:35.302931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/676] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.304124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.317443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:35.318621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:35.383789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/677] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:35.383957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810c84b-4bd5-42e6-8dc5-6807a196cac7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.384595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.396242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.418491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.419119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.419399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.419700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.426193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.427638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:35.439077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:35.440199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:35.441281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:35.443723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 339/678] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:35.444718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.486327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.494922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40979f9-10bf-41f3-b3ab-c30d8c95caba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.529771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.530376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.530658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.530933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.531692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=337f5152bd51454cb58d6b2092d5cf8b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.532004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.533417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.539030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/679] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/role_assignments?user.id=337f5152bd51454cb58d6b2092d5cf8b&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.539829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.558404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f8326-b04a-4e02-8fd4-e22a7c0470aa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.602288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.603238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.603647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.604387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.605377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.605829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.607413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.627489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 340/680] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.627489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.655124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2269eb-767f-4857-ab69-2a8bd96bb1e1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.658336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.694997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.695731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.696074 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.696407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.697115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.697480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.699558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.715055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/681] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.715055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.735776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.743274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0da9e0-3e40-4f5c-a6e4-d558c2e6d8c8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.782832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.784246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.784606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.784952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.798705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:35.805105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:35.806236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:35.807272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 341/682] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:35.808041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.833996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d0d1c-c217-43c3-a96e-83fa8c3c75b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.881217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.882088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.882371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.882648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.883342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.883760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.885182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.897105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/683] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.897726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.910648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:35.925302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d311a23-1ff7-4b74-b151-d8b2f6b7d96d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:35.964227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:35.965093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:35.965450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:35.965894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:35.966857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=6c7d784c96b044ef8b383767e8c1cb08) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:35.967237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:35.968897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:35.983637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 342/684] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/role_assignments?user.id=6c7d784c96b044ef8b383767e8c1cb08&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:35.983637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:35.986955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:36.016766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdf639-1c36-422f-b294-daa68eccb9d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.057612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.059015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.059299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.059728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.069206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.071042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:36.088717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:36.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:36.091969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:36.094956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/685] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:36.096079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.111438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965ae3-071c-4beb-8cfc-ac772a176cd7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.150730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.151700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.152228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.152510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.153191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:36.153500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.154870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.166711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 343/686] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:36.168229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.186906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:36.229090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2db5c-2393-4ef1-b833-86e5f43c0015 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.284972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.285603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.285893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.286172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.287198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=476a1946148e4758b071e9b6ce788bcd) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:36.287516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.289688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.294271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:36.295550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160387fa-96b0-4639-90ee-8c44ee27aca7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.303650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/687] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/role_assignments?user.id=476a1946148e4758b071e9b6ce788bcd&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:36.303650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.386720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.387445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.387755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.388038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.388819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:36.389126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.393581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.428287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 344/688] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:36.438270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.499258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd6a04-c3a6-4268-9e3e-1368af7b4337 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.540596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.541174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.541457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.541730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.553342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.553907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:36.555351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:36.571386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:36.572427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:36.573307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:36.573307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:36.573307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:36.573307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/689] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:36.578949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.607010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff630-74f9-4300-89c4-8725db3b7467 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.650593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:36.652600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.653678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.654193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.654789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.717402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f14968-fa32-4a01-8cdc-f66f68645bdf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:36.736071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.737921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.774454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:36.775077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:36.775358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:36.775691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:36.776349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:36.776652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:36.778706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:36.807023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/690] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:36.813026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.878265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:36.909577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 345/691] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:36.914794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:36.979781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e72727-0af6-4e74-a586-5b57b6b886c6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.001222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:37.041086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.041733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.042015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.042286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.057119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.058607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.065333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b1207-2278-46e5-88c1-067f0e4ad965 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:37.083169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:37.084357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:37.085567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/692] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:37.091856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.134771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.135418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.135789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.136071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.153440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.154911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:37.181706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:37.182891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:37.184766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:37.184766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:37.184766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:37.191107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 346/693] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:37.192779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.227632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:37.259361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d4b2-4ad5-4729-9b0a-33172846a1a7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.297093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.297972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.298400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.298817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.299913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:37.300270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.301852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.331705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/694] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:37.331705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.361859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:37.384775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2bfb1-e512-4663-b5e1-4460deded6f3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.439813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f5633-71f8-4b32-bb32-d17f41580f71 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.466196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.467169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.467685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.468190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.469056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:37.469571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.471139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.484589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.485215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.485494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.485769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.501309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 347/695] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:37.502849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.601840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.603206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.634310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:37.683458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:37.710393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f22875-54b9-45c9-8fee-8d80e1b24914 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.719431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/696] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 06:50:37.720827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.747303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.747960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.748229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.748499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.755714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.757154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:37.768963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:37.769882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:37.770757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 348/697] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:37.785965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.837240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38504557-ca4c-43b4-afb1-a41276a65e0a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.868555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:37.880460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.881115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.881397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.881669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.893469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.895042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e22c987c-101e-4bba-8526-c2f59d037336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:37.909892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:37.910862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:37.911761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:37.911761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:37.911761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:37.915506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/698] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:37.916833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070ed0-611d-4115-a43f-3289494df1fe None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.917116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070ed0-611d-4115-a43f-3289494df1fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.917391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070ed0-611d-4115-a43f-3289494df1fe None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.919385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/699] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:37.920297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070ed0-611d-4115-a43f-3289494df1fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:37.951195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d818d6a-d91d-407b-bf60-e6683998c744 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:37.989887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:37.990559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:37.990943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:37.991267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:37.992098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:37.992409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:37.993902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62feafa0-e4fc-4283-981f-59a8e249f12c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.008563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 349/700] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:38.009818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.010150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.010435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.018562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.032485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:38.033700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:38.038998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070ed0-611d-4115-a43f-3289494df1fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.074645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.075265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.075549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.075851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.076493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:38.076793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.078161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.095236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/701] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:38.096182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.112812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 350/702] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:38.113602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.140963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.201893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915319f-0faf-4934-b2a0-4d0a5e9bfc51 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.237706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.238361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.238637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.238910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.239701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=8f3a4e99f3044d35aa2e310478f3653b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:38.240011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.241421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.246532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db670b-c05c-4d22-afa0-4e2adf7b1c34 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.254196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/703] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/role_assignments?user.id=8f3a4e99f3044d35aa2e310478f3653b&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:38.255182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.288594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.289252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.289534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.289812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.291957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.300061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.301705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:38.313996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:38.316213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:38.318361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 351/704] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:38.323638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.414998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d97d1-9a21-41ba-9122-c4148c65404b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.424285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.445595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8115b0-2fe1-4fa0-b8ef-7cd93020ace1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.456601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.457624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.458109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.458611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.470538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.472470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6cbba0e2-1125-42e7-97e9-a131099e4b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:38.489077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:38.490180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:38.491114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:38.491114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:38.491114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:38.491313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.491313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.491313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.491313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.491566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:38.492221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.493132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d90fac-bf51-418d-9292-6238beb627c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.494127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/705] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:38.495631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64581-8e3e-4e08-aac7-fa94abcbd24e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.496674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64581-8e3e-4e08-aac7-fa94abcbd24e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.497385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64581-8e3e-4e08-aac7-fa94abcbd24e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.498844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/706] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:38.500043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64581-8e3e-4e08-aac7-fa94abcbd24e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.507474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 352/707] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:38.514255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.514560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.514818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.533978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:38.535183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:38.605826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.628550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64581-8e3e-4e08-aac7-fa94abcbd24e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.631736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 353/708] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:38.632810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.685390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.685988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.686276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.686653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.687310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:38.687637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.689040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.703002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/709] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:38.703960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.729054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.758722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf48953-9d6f-4bb1-8301-9f66ea3d85a2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.803663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:38.804279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:38.804560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:38.804833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:38.805627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=d80a908c1da74b3fb3e82c8277556e81) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:38.805857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:38.807366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:38.819444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250fa1f-9c8f-45f0-9dc9-7fd4147cc63e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:38.819773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 354/710] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/role_assignments?user.id=d80a908c1da74b3fb3e82c8277556e81&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:38.820728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:38.915364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:38.981768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065b63-dd7a-4c4c-a0d6-5aa902bad063 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:39.031031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.031888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.032323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.032520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.032797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.033066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.033454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.033645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.034111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.034965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.035068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.036687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-34cc0fe7-c197-49fb-aa12-440df547323f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:39.046170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:39.047202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:39.048176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/711] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 9 06:50:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:39.048955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76db8c7-dbd5-4139-9ecf-252d10696b66 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.049543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76db8c7-dbd5-4139-9ecf-252d10696b66 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.049543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76db8c7-dbd5-4139-9ecf-252d10696b66 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.055316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/712] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:39.056505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.056801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.057068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.067324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 355/713] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:50:38 2022] POST /identity/v3/users => generated 254 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:39.068505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:39.092349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:39.095560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:39.204639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/714] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:39.238016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:39.258760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:39.275819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218bafc-3005-4dc9-a5a8-3d389160e65f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:39.284543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:39.322975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.324163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.325168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.325889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.326990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:39.327494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.329815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.337489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 356/715] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 9 06:50:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:39.343677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:39.379341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655e0ac-de2d-4134-a377-14a1040a8bb1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:39.418281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.419187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.419533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.419925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.420574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.422003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:39.431467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:39.432503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:39.434031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:39.439037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/716] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:39 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:39.439921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:39.473573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8b874-41f3-4987-bd72-d7b7ffac3594 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:39.512524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.513184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.513461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.513731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.514370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:39.514716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.516164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.524745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:39.529595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 357/717] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 9 06:50:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:39.551655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7dd91-87cd-4984-b4f6-14e7d4dc98c8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:39.590603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.591621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.592128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.592641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.593476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:39.593977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.595536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d74b1-962f-4506-8d22-ccc0dd0937ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.601282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/718] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:50:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:39.607198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:39.674966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:39.769045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ebde2-af07-4322-a5d4-c4710cc38620 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:39.804827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:39.805446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:39.805723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:39.805992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:39.806631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:39.806931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:39.808357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01d610-61f2-40ba-a4a5-bf6ffcf89dc0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:39.813356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 358/719] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:50:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:40.729634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcabb00-f16e-49d6-a661-1ea5427e38f8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:40.730701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcabb00-f16e-49d6-a661-1ea5427e38f8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:40.730701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcabb00-f16e-49d6-a661-1ea5427e38f8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:40.732319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/720] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:40.737457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:40.737747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:40.738025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:40.760901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:40.762469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:40.849021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 359/721] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:40.887854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcabb00-f16e-49d6-a661-1ea5427e38f8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:41.039340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcabb00-f16e-49d6-a661-1ea5427e38f8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:41.082769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:41.083405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.083708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.083987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.094887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:41.099798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:41.100695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:41.101526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/722] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:41.103897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:41.107544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:41.154680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:41.259343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee04d92-ab52-4be4-b87d-963fbed70dca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:41.312131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:41.312689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.312967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.313242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.313938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:41.314240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:41.315688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:41.334155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 360/723] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:41.340328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:41.471144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc899f6-991e-4861-b577-3adc8d2bfbf8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:41.507059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:41.507725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.508006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.508278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.523638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:41.525081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-501fdf09-4655-47fb-bf5c-560e55344083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:41.549673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:41.550637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:41.551511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:41.551511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:41.551511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:41.551511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/724] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:41.556470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:41.589223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b181c5e-5fa0-4dcd-adc9-18ee087d6860 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.590028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b181c5e-5fa0-4dcd-adc9-18ee087d6860 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.590398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b181c5e-5fa0-4dcd-adc9-18ee087d6860 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.592085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/725] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:41.597014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.597299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.597574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.622108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:41.623306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:41.711739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:41.720521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:41.734552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/726] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 06:50:41.740080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09422e8a-072d-4a0a-9784-9bf86b402122 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:41.759271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:41.777781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:41.778380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.778655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.778922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.779558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:41.779885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:41.787632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:41.795391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 361/727] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:41.801911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:41.945314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698d71c-d032-4fc9-a248-29f05acb367e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:41.984854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c64f0-1c8c-4afe-bafb-c69b216e2744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:41.985913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c64f0-1c8c-4afe-bafb-c69b216e2744 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:41.986394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c64f0-1c8c-4afe-bafb-c69b216e2744 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:41.986866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c64f0-1c8c-4afe-bafb-c69b216e2744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:41.999068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c64f0-1c8c-4afe-bafb-c69b216e2744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2e8c64f0-1c8c-4afe-bafb-c69b216e2744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:42.004705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:42.005599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:42.006427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:42.014335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/728] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:42.016351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1928e-9de8-4023-b7fd-3b1876e40f60 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.016967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1928e-9de8-4023-b7fd-3b1876e40f60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.017445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1928e-9de8-4023-b7fd-3b1876e40f60 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.019306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/729] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:42.020424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1928e-9de8-4023-b7fd-3b1876e40f60 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.061650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47924a-ed6c-4fed-adcd-faeff54a816a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.066812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.096642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.097278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.097609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.097883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.106554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.108266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7bec49d4-25b9-4015-80d9-1d89176fc892 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:42.120232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:42.121193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:42.122118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:42.129027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 362/730] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:42.129154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1928e-9de8-4023-b7fd-3b1876e40f60 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.129264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.129369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.129648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.148267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:42.149866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:42.152872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.166517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.167123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.167440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.167752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.168425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:42.168726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.170087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f537788-fc8d-4fd9-82f6-b63e2cd93292 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.178149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/731] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:42.179310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dd7da-9af7-4292-a31d-6d6a9f499d13 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.179609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dd7da-9af7-4292-a31d-6d6a9f499d13 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.179896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dd7da-9af7-4292-a31d-6d6a9f499d13 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.181216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/732] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:42.181755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1dd7da-9af7-4292-a31d-6d6a9f499d13 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.245806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 363/733] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:42.247584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.294407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1dd7da-9af7-4292-a31d-6d6a9f499d13 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.297628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.331536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.332278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.332561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.332913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.333645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:42.333956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.335375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd7401-8235-4b00-95b2-50f45f17a4ec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.353750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/734] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:42.354984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6d75e-0eab-41fc-ab12-da9e00297d2b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.355405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6d75e-0eab-41fc-ab12-da9e00297d2b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.355713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6d75e-0eab-41fc-ab12-da9e00297d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.363018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1325f-8c96-4ee7-991a-6b0b031f8f01 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.373075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-91a6d75e-0eab-41fc-ab12-da9e00297d2b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:42.374239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-91a6d75e-0eab-41fc-ab12-da9e00297d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:42.398356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.398987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.399263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.399533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.406552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.410597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.412071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:42.426203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:42.427337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:42.428245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:42.428245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:42.428245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:42.428245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 364/735] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:42.428245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.445586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/736] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:42.446820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b17f2-0983-4e42-99d6-0933cf723ebb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.447111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b17f2-0983-4e42-99d6-0933cf723ebb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.447389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b17f2-0983-4e42-99d6-0933cf723ebb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.451687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/737] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:42.451687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b17f2-0983-4e42-99d6-0933cf723ebb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.528319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14028802-99f3-452a-ae79-569e297e9d6c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.554354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.562097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.562720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.562997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.563263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.564976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b17f2-0983-4e42-99d6-0933cf723ebb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.573117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:42.578432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:42.579377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 365/738] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:42.580675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.604351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.604970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.605456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.605818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.630477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.673080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.674805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.678864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74762-d7a2-4906-83a1-ae439e3ef35f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.711203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.711834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.712111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.712383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.713016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:42.713318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.715190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bdb2d-44ad-4836-b93b-09d645ff31af None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.726474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 366/739] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:42.727721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.728004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.728282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.745923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:42.747212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:42.763268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.773375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/740] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 06:50:42.774147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.814518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 367/741] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 06:50:42.815351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.860430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:42.881714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad4019-a004-41b1-998c-41fe7a23abb3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.915044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.915682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.915968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.916244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.916879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.918851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.919869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21bab8-a6f6-4040-874c-b8b37c5cbbb5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:42.934860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/742] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:50:42 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:42.935931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:42.975823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:42.976471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:42.976752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:42.977025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:42.982483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:42.982802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:42.984242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:42.998849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 368/743] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:42.998849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.000713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.055831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e3aea-ef6c-4176-aee7-b9dddb318d8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.100176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.100176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.100449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.100665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.110492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.112014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.117689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.122445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4423ca4-c805-446d-91ec-446600c1dcd8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:43.125932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:43.126947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:43.128250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/744] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:43.128832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.160021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.160661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.160938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.161201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.172223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.174047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:43.189657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:43.190642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:43.191513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:43.191513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:43.191513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:43.191513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 369/745] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:43.191513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.241717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac2bd-2f6b-46a8-a4f9-fe385860fb32 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.255730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.280114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.280727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.281005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.281278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.281924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.283566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.299694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/746] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:50:43 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:43.300511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.312629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae303f6a-4441-40ba-9f31-5631a5f1ee93 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.361165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.362193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.362782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.363122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.377166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.379045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.388294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:43.398007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:43.399004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:43.399922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:43.399922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:43.399922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:43.404087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 370/747] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:43.405215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.445931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6851ca-27c6-461a-adf6-df32eaafcaf2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.486915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.487536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.487862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.488198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.489006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:43.489312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.490781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.515720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/748] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:43.516404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.522067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63272563-48ba-4d62-8692-df9e6154aa5f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.558120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.561627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.562425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.562862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.563274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.564166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:43.564467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.565830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.579173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 371/749] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:43.584314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.648549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c0a87-cfed-466b-900b-181bab93cd5b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.651505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.695206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.696435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.697229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.697700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.698813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:43.699317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.701743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.707212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f70e7e-dd11-4e59-951f-1f962334e1e6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.721013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/750] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:43.721013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.756297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.757338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.757715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.758498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.772465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.774113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:43.794502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:43.795542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:43.796475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:43.796475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:43.796475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:43.796475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 372/751] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:43.797972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.846252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.859508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03269a6-da8c-45c8-987c-dea1fcf60c7d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.897406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.898415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.898902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.899401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.907452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.909209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:43.921648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:43.922598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:43.923481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:43.929556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/752] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:43.929556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:43.938410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50641ceb-8020-4f9b-b916-2d9b17a1f393 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:43.950593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:43.978318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:43.979220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:43.980389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:43.980779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:43.989419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:43.991094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:44.004161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:44.005251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:50:44.006193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:44.010723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 373/753] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:44.011793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.046304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fa0e3-4894-45ac-ae51-6591ed4b8898 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.080996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.081942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.082440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.082890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.083770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:44.084264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.085850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.100519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/754] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:44.101788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.114966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:44.126718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca8c6-ff84-42a0-bc68-440854b9d056 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.166153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.167121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.167913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.168340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.169350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:44.169814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.171483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.190829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 374/755] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:44.192034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.208873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c221867-4307-4d47-954a-8f1aea3f4a86 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.243260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.243946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:44.244102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.244789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.245137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.246717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:44.247088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.248676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.270958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/756] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:44.270958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.311946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55fe57-2cbe-45ca-8aa9-54f3756cc975 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.335396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:44.350597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.351667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.352305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.352759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.360789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.362777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.372190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538df74-91db-4ae6-b6d2-33dca1beaa58 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:44.376734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:44.377873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:44.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:44.381442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 375/757] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:44.382621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.408088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.408785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.409137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.409516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.410277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=8f3a4e99f3044d35aa2e310478f3653b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:44.410777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.412192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.423007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/758] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/role_assignments?user.id=8f3a4e99f3044d35aa2e310478f3653b&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:44.423007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.500183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:44.508190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce5bff-3acb-4f35-8c2b-50d1fb83dfa2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.556579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.563415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.564007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.564503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.609003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415789f-4b7f-421e-8881-4091ced6456b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.645617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.647423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.669754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:44.679886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.680508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.680789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.681064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.681760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:44.682068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.691630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.698024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/759] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:44.703983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.833523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:44.834196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940c6bd-d3a7-4c5d-987a-327c550c545c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:44.873924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1b6b-a0ad-4388-8d9e-fc177beba4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:44.875034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1b6b-a0ad-4388-8d9e-fc177beba4a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:44.875366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 376/760] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:50:44.875559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1b6b-a0ad-4388-8d9e-fc177beba4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:44.875936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1b6b-a0ad-4388-8d9e-fc177beba4a1 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/d80a908c1da74b3fb3e82c8277556e81/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:44.878349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e1b6b-a0ad-4388-8d9e-fc177beba4a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=d80a908c1da74b3fb3e82c8277556e81, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:44.879918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e1b6b-a0ad-4388-8d9e-fc177beba4a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:44.881361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:44.900950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/761] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:44 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/d80a908c1da74b3fb3e82c8277556e81/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:45.033177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede3bb-9229-4b99-951c-8200adb957ba None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.033538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede3bb-9229-4b99-951c-8200adb957ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.033760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede3bb-9229-4b99-951c-8200adb957ba None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.034145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29856319-87fc-48f7-ae5c-152e05a61b04 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.035493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/762] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:50:45.051640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.051972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.052231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.091783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:45.092921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:45.098615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.099266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.099549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.099994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.113661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.115286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:45.136997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:45.138379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:45.139960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:45.139960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:45.139960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:45.139960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 377/763] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:45.143993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.153549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:45.178338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/764] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:45.204042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.209134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:45.312209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e04bf9-8021-4081-9a7d-2e25fa2a8c65 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.326901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8276c5-91f0-49e3-8b08-b3e157222481 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.364505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.365513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.365937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.366886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.367681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.369329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:45.380419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:45.381609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:45.382626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:45.385299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/765] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:45.391720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.416869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.417601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.417819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.418091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.418747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:45.419045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.420441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.446881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 378/766] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:45.447976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:45.456751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.529059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22812da-78d6-4918-a4ac-a1c38e212a01 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.579444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.580123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.580652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.580928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.582036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:45.582480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.584218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7beb1-78bf-4da4-8685-eb1a7f75ca15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.590922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/767] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:45.592863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8d23f-7610-4338-b3c3-ab21632405d7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.593145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8d23f-7610-4338-b3c3-ab21632405d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.593682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8d23f-7610-4338-b3c3-ab21632405d7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.595466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/768] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:45.600437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8d23f-7610-4338-b3c3-ab21632405d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.601922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130d224-5e26-41f9-a752-416711dd80f8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.603740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:45.640685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.641533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.641874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.642263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.649632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.651123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4e8d9788-691f-41c2-95e6-1e79848c800f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:45.664158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:45.665206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:45.666304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:45.668086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 379/769] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:45.670618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.670977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.671319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.689944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:45.691432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:45.710375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8d23f-7610-4338-b3c3-ab21632405d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.722047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:45.747629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.748261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.748538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.749176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.749840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:45.750512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.751909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.760235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/770] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:45.762101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.770014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 380/771] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:45.771132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.906173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:45.916929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880126-6e52-418e-a2e9-654db51f65fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.918118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701435d7-be96-4127-9be0-bf89529acbd5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:45.955991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.956649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.956934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.957214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.958175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:45.958490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.960719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.965796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:45.966671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:45.967158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:45.967645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:45.968556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] RBAC: Adding query filter params (interface=public, service_id=f18f38a90d394cd6b44f6acbc4a91bdb, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:45.969068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:45.971394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:45.976119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/772] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:45.977063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:45.983760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 381/773] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/endpoints?service_id=f18f38a90d394cd6b44f6acbc4a91bdb&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:45.985006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:46.023134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:46.115270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298e161-ef95-4ad6-8f5c-29046a84acc5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.117224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec660a5-daac-4d1c-9b01-cfeeb41f512f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.163628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.163628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.163628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.163628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.164326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=4dd6c53afc054e2384dc0d247a9b935c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:46.164326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.164326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.170609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 382/774] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/role_assignments?user.id=4dd6c53afc054e2384dc0d247a9b935c&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:46.171681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:46.173421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.174194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.174767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.175053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.176765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.178201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:46.191762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:46.192792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:46.193780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/775] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:45 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:46.200169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:46.201179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:46.322464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:46.329016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eec1f-8eed-4b77-b1f2-5eb55e42dd2c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.369909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.371303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.371842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.372123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.374465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:46.374465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.374857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.381179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/776] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:50:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:46.388369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:46.410551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9643-6313-44b3-b162-bdedf926267a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.446071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.447004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.447506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.448039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.449483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.451114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.457041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 383/777] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:50:46 2022] GET /identity/v3/services => generated 772 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:46.514073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b4d6-16d1-45a5-b1fe-494b54beb772 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.554689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.555324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.555622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.555929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.556585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:46.556953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.559089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.565436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/778] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:50:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:46.571190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:46.701364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:46.719246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834716-a271-45e1-adb2-734d767bab0c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.756264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.756873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.757199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.757474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.758171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] RBAC: Adding query filter params (interface=public, service_id=bbf0bf9e8c1240beaaf1f96d1a7471ae, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:46.758474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.759851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610de15-a740-43d7-ad9a-bb274e55beb7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.766045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 384/779] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:50:46 2022] GET /identity/v3/endpoints?service_id=bbf0bf9e8c1240beaaf1f96d1a7471ae&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:46.771630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:46.806706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:46.904450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd0f69-6a30-4f2f-bc4d-b6c2420c6dd2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:46.949880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7f795-3ecd-4eed-8b75-1b2f62be338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:46.951175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7f795-3ecd-4eed-8b75-1b2f62be338c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:46.951681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7f795-3ecd-4eed-8b75-1b2f62be338c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:46.952125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7f795-3ecd-4eed-8b75-1b2f62be338c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:46.952886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7f795-3ecd-4eed-8b75-1b2f62be338c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:46.954406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7f795-3ecd-4eed-8b75-1b2f62be338c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:46.964100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/780] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:50:46 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:47.058283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc9d24-06cb-4691-a002-fe806d287922 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.058773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc9d24-06cb-4691-a002-fe806d287922 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.059147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc9d24-06cb-4691-a002-fe806d287922 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.060890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 385/781] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:47.066129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fdb18-262e-46b8-8daf-2b04812f5048 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.066645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fdb18-262e-46b8-8daf-2b04812f5048 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.067098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fdb18-262e-46b8-8daf-2b04812f5048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.096749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7e1fdb18-262e-46b8-8daf-2b04812f5048 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:47.098279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e1fdb18-262e-46b8-8daf-2b04812f5048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:47.197012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/782] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:47.209994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ee3df-8221-40c3-a4d9-75d6e7251bcb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.210466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ee3df-8221-40c3-a4d9-75d6e7251bcb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.210466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ee3df-8221-40c3-a4d9-75d6e7251bcb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.212965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 386/783] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:47.217604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.218072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.218522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.223849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ee3df-8221-40c3-a4d9-75d6e7251bcb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:47.244328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:47.246395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:47.265320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:47.341408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/784] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:50:47.352347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ee3df-8221-40c3-a4d9-75d6e7251bcb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:47.365170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:47.416564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:47.417752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.418031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.418309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.440543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:47.445221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:47.446172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:47.447011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 387/785] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:47.453225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:47.470999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:47.537900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaf49d-48d4-4917-84f6-4fb1d11fcc40 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:47.601555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:47.602192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.602474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.602745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.626928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:47.633601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:47.634614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:47.635452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:47.639629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/786] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:47.644142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be36e0a-e566-486e-8a90-2583305c1878 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:47.646473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:47.691625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:47.720459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:47.721136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.721361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.721637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.722310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:47.722611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:47.724010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:47.745018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 388/787] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:47.749885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:47.811349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165648e2-797c-44fb-8449-02befa9a6567 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:47.879957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:47.881625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:47.882656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.882656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.882882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:47.883624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:47.884165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:47.885535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:47.905901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/788] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:47.909246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:47.944972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ce3b-22f1-4b8c-85dd-0adc20a8907b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:47.987356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:47.987938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:47.988217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:47.988488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.001521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.003291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:48.019035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:48.020005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:48.020909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:48.020909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:48.020909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:48.020909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 389/789] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:48.026912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.067982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:48.091805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74456a-f7ec-4a2d-844b-a2cca2b8537f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.130991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.131652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.132834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.133110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.155374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.157603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:48.182644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:48.184966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:48.186173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:48.186173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:48.186173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:48.189198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/790] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:48.190472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.239834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0328e-1e67-4334-bb60-49a9a562893b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.293182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd0f01-c4dc-431d-9ebb-db50d0a69624 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.294132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:48.320013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:48.320304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.320906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.321187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.321468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.322115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:48.322420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.323856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.329110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.329726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.330008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.330282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.330924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:48.331227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.333712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.345814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 390/791] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:48.350103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/792] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:48.351216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.353492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.498821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0d91c-d295-4365-9a60-75e0bbf0403f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.534881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.535575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.535888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.536208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.544526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.545976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:48.559548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:48.560539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:50:48.561468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:48.565285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/793] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:48.566463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.568400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24586ce8-fc08-4fae-9aef-b6f308fb3fd4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.625546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:48.629817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.630872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.631395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.631888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.647694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.649446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:48.675861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:48.676848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:48.677789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:48.685233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 391/794] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:48.691916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.737052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d06dd-ed6b-460c-a85f-0b37e418aba1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.747539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:48.775893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.776491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.776798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.777075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.777808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:48.778115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.779926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489fafe-b16a-463e-8bce-00ae83361f76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.803740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/795] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:48.805392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef58887-8cf6-456e-b9f4-6c3a99bdf8fa None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.805679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef58887-8cf6-456e-b9f4-6c3a99bdf8fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.805958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef58887-8cf6-456e-b9f4-6c3a99bdf8fa None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.811719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/796] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:48.812776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef58887-8cf6-456e-b9f4-6c3a99bdf8fa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:48.828422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b5194-6a51-4c5c-af98-e7119bc13be1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.882776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.883398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.883710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.883991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.884725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:48.885031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:48.886705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c866b8-75c3-4522-9391-1ef210bfae78 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:48.911267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 392/797] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:48.913045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.913517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.914072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:48.934478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:48.936187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:48.951066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef58887-8cf6-456e-b9f4-6c3a99bdf8fa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:48.953211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:48.998583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:48.999227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:48.999504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:48.999809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.030972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 393/798] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:49.032247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.068909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.095403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.096900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.151241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06fcaf-ff16-4418-94af-f972361cb17e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.187214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.187884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.188180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.188455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.199474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.201022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.206130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/799] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:50:49.206130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.206130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e8861ac6-9d2d-48f4-9095-cf0ec2baec1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:49.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:49.222851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:49.223757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:49.223757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:49.223757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:49.223757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 394/800] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:49.224382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79d234-1de2-460e-b4ef-eb95a5304a36 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.224830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79d234-1de2-460e-b4ef-eb95a5304a36 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.225112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79d234-1de2-460e-b4ef-eb95a5304a36 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.226486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 395/801] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:49.227032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79d234-1de2-460e-b4ef-eb95a5304a36 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.315270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.316418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb67be-56c2-4dd0-9446-0216bdb5db4a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.337019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79d234-1de2-460e-b4ef-eb95a5304a36 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.354564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.356168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.356696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.357113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.367850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:49.372901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:49.373121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:49.374216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:49.375238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:49.376022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.376022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.376022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.378056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/802] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:49.379048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.389160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.390803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7aefd548-0adc-428c-adf6-a733c212cd23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:49.409931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:49.410896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:49.411840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:49.411840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:49.411840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:49.415925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 396/803] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:49.417737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.418046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.418300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.436287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:49.437487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:49.446611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.485586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c8e07-4095-4594-8ffa-ebcade10b516 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.504524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 397/804] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:49.505353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.525038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.526060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.526489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.526778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.527519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:49.528364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.529732 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.541194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/805] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:49.541997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.582925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.620980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c94ce2-42bf-4320-93a7-8295f005de1c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.639462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208092-75cf-4ac4-b720-acb23a9177ff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.654921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.655623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.655922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.656204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.656869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:49.657170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.658538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38561b-4e54-4840-9d43-e1d54441543f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.661343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.671670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 398/806] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:49.671670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80701b81-09d1-4e03-bc7f-3c2b5ebb7b1d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.671670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80701b81-09d1-4e03-bc7f-3c2b5ebb7b1d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.671670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80701b81-09d1-4e03-bc7f-3c2b5ebb7b1d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.671670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 399/807] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:49.671670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80701b81-09d1-4e03-bc7f-3c2b5ebb7b1d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.678251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.678877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.679159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.679432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.680113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:49.680415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.681773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.694292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/808] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:49.695075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.772500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80701b81-09d1-4e03-bc7f-3c2b5ebb7b1d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.794107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a65e-46f8-4f49-af5f-698fea3ed920 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.798292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.806671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.807298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.807580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.807913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.808490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.809849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:49.820399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:49.821540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:49.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 400/809] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:49.824195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.831480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.832136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.832413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.832680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.840486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.841865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-32f64ad6-f4f2-4437-8fdf-69e6f40a73a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:49.852907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:49.853955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:49.855091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/810] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:49.855891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.856174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.856446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.874800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:49.876021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:49.909741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:49.935276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec8e7f-c2b3-4bf8-9aa3-e3a932457ab8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:49.941040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/811] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:49.941855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:49.983791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:49.984421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:49.984699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:49.984964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:49.994732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:49.996726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:50.011386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:50.012612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:50.013500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:50.013500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:50.013500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:50.013500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 401/812] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:50.013500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.039484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9245-f005-4553-a399-b27f8fedbc9a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.058905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.074329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.074962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.075242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.075521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.082032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.083456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:50.095587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:50.101691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/813] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:50.102798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.125159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecc692-d5cd-4773-bf92-6a0395a5361e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.128522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.158564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.159160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.159436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.159735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.160356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.160654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.162003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.166274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 402/814] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.167419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.196617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab70ab-0da9-49c2-9101-e32b1c977f14 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.230610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.231213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.231491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.231795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.232478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.232786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.234198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.245949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/815] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.246853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.269777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac444a4-5f50-438c-ac5f-7bdc838756a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.277472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.303923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.304543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.304818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.305087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.305660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.307003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:50.315478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:50.316936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:50.317776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 403/816] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/services/network => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:50.318555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.338153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.348574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b84a-3256-41f0-b8c4-4d1b329611a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.383022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.383660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.383945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.384217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.384857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.385161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.386639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.397542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/817] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.398336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.420662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525d1f-688a-42e5-aeab-058c5ecf78c2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.454609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.455214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.455493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.455804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.456479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.456782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.458223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.470313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 404/818] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.471063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.499539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.500420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f35122-46a5-4358-ba6e-2938e4b8d587 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.535456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.536097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.536379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.536659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.537288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.537590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.538949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.543687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/819] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.544470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.564653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.572201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad68dc-6038-4046-a6fb-558c27c07879 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.606850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.607438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.607756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.608040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/8f3a4e99f3044d35aa2e310478f3653b/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.609939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=8f3a4e99f3044d35aa2e310478f3653b, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.611374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.624649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 405/820] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:50 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/8f3a4e99f3044d35aa2e310478f3653b/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:50.625450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.663165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9cd07-600d-41f0-9b8c-62d6ffef0a14 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.703591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.704277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.704630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.704914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.705600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.705912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.707326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.717806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/821] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.717806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.733419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.747924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96a974-0efb-4a9c-8208-ded2c81214d8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.793849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.794489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.794772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.795032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.802437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.803905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.814587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:50.816884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:50.822083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 406/822] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:50.823317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.846269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad38ae1-8283-47a9-81c8-2bb4342bb69a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.888763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.889473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.889635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.889906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.890628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=d80a908c1da74b3fb3e82c8277556e81) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:50.890933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.892456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:50.907776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/823] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/role_assignments?user.id=d80a908c1da74b3fb3e82c8277556e81&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:50.907776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:50.948957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838393d1-6df0-4575-bfe4-dbff76b99594 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:50.988342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:50.989269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:50.989551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:50.990252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:50.991188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:50.992618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.027763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 407/824] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:50:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:51.028827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:51.050629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a107650-e78d-4a29-867b-3434828ca764 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:51.054304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:51.132135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:51.133454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:51.133942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:51.134919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:51.135896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:51.136437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:51.138461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.148146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/825] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:50:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:51.156695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:51.168793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:51.198734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c86607-9419-4556-af4d-1383b20f0933 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:51.248979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:51.249617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:51.249897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:51.250169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:51.250926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:51.251229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:51.253112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.274206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 408/826] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 9 06:50:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:51.279298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:51.325452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33754f36-5932-4f7d-8666-5ed5816df0b8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:51.346951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:51.366875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:51.367930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:51.368451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:51.368958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:51.370014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:51.372266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.388298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52ff8-154a-4480-8465-e9925478df6e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:51.398822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/827] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:50:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:50:51.437992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:51.438644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:51.438925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:51.439248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:51.452003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:51.453544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:51.471774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:51.472812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:51.473758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:51.473758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:51.473758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:51.473758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 409/828] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:51.478880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:51.583134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:51.643243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5b78-b603-4bfc-b99d-97b12fc325f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:51.680518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:51.681504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:51.681988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:51.682474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:51.683516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:51.684087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:51.685667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.699378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/829] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:51.704776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:51.820878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25c90-4883-4059-8e99-d0e8dced09fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:51.838663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:51.872163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:51.872810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:51.873093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:51.873369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:51.882135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:51.883651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:51.897139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:51.898032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:51.898915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 410/830] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:51.904573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:52.024982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19f63-a951-476b-93ef-fe5453367ab1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:52.046356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:52.062745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:52.063359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:52.063723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:52.064000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:52.064690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:52.064995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:52.066414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df773fed-dddb-4b4c-a133-06ad5c8a4387 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:52.078774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/831] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:52.084653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:52.185390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782614e3-1fa0-49dd-82a9-7935240845d8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:52.223676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0ee0f-272d-464a-98c5-832355265434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:52.224578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ee0f-272d-464a-98c5-832355265434 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:52.225032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ee0f-272d-464a-98c5-832355265434 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:52.225450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ee0f-272d-464a-98c5-832355265434 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/4dd6c53afc054e2384dc0d247a9b935c/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:52.241197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0ee0f-272d-464a-98c5-832355265434 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=4dd6c53afc054e2384dc0d247a9b935c, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:52.242855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0ee0f-272d-464a-98c5-832355265434 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:52.258325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 411/832] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:50:52 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/4dd6c53afc054e2384dc0d247a9b935c/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:50:52.828663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86d95-6759-4c5f-9ffa-e73474c330ae None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:52.829688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86d95-6759-4c5f-9ffa-e73474c330ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:52.830170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86d95-6759-4c5f-9ffa-e73474c330ae None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:52.831985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/833] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:52.837382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:52.838223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:52.838695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:52.860031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:52.861244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:52.879561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:52.933948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 412/834] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:52.957296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be86d95-6759-4c5f-9ffa-e73474c330ae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.066388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be86d95-6759-4c5f-9ffa-e73474c330ae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.101734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa8af7-9467-4db9-8f50-7a858ea7a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.102342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa8af7-9467-4db9-8f50-7a858ea7a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.102629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa8af7-9467-4db9-8f50-7a858ea7a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.102902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa8af7-9467-4db9-8f50-7a858ea7a9d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.114951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa8af7-9467-4db9-8f50-7a858ea7a9d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-85fa8af7-9467-4db9-8f50-7a858ea7a9d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:53.119756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:53.121352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:53.122196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/835] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:53.128165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.231135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b76b24-e15e-4565-84dc-3af41f8062fd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.233883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921e31-6c61-4804-85e2-fc35e3043236 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.234171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921e31-6c61-4804-85e2-fc35e3043236 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.234467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921e31-6c61-4804-85e2-fc35e3043236 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.236821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/836] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:50:53.244751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.245648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.246112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.266059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:53.267490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:53.273015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.273897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.274380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.274877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.275783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:53.277021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.278768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.295413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 413/837] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:53.296384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.310511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:53.316537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:53.350228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/838] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:50:53.373736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.420560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543f6c4-bf68-42af-a8dd-5fa8b994c761 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.456821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.457463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.457745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.458019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.469638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.471094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.483503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce358-b2ee-4639-b37c-6344bb7d9793 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2eeb8b30-816c-447e-8a71-5b453a495975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:53.486214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:53.487172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 414/839] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9840c8-0ee1-44fa-9dd0-4812f1822f55 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9840c8-0ee1-44fa-9dd0-4812f1822f55 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9840c8-0ee1-44fa-9dd0-4812f1822f55 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.492927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 415/840] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:53.495735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9840c8-0ee1-44fa-9dd0-4812f1822f55 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.521814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.522470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.522749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.523186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.523617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.524987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-80a14c2f-aa5f-4623-bab7-0539124db0d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:53.534252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:53.535197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:53.536174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/841] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:53.536961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.536961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.537080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.554590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:53.555794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:53.571369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:53.585114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:53.613652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9840c8-0ee1-44fa-9dd0-4812f1822f55 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.631707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/842] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:53.631707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.655433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.656347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.656772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.657189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.658008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:53.658578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.659973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.677282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 416/843] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:53.677282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.742814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfe6bd-8037-4941-9154-cd27b5ff3cc4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.779157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508aab0-c54f-426b-ad8f-e6504a1df3f5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.781092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.781822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.782109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.782386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.783016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:53.783321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.784717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.794122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/844] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:53.794781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.805487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:53.816419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.817046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.817325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.817645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.818180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.819536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.825115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:53.829958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:53.830948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:53.836687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 417/845] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:53.837462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.913359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35548a-4d8c-41c9-92a6-1e91af9369a8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.949169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4ab6e-4c36-4678-b063-cd030f2443c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:53.955456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.956119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.956717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.956717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.964748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.966297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:53.977959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:53.978917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:53.980480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/846] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:53.981071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:53.992330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:53.992943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:53.993225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:53.993496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:53.994140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:53.994442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:53.996030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.012019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 418/847] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:54.012019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.075050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.124062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.134767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224c83-0d5f-4410-8dab-a794b4b938ed None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.148458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdebe8-7979-419c-b675-6f12815303aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.177267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.177865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.178148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.178425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.179165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:54.179475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:54.180941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.202796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 419/848] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:54.203058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.207457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.207457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.207457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.207457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:54.207457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:54.207457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.209041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.215990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/849] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:54.221261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.328281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af3eb7-175a-443a-839f-58313e40db17 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.342924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.343070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8219d-22f2-475b-9881-d1c03527125d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.366074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.366682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.366959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.367230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.381627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.382600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.383167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.383449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.384161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:54.385584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:54.387313 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bd12f-3e99-49f5-8e0d-16edeec64b40 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.399668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/850] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:54.399668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4890e7-0a6a-44b1-acb2-76ddbf521ef4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.399668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4890e7-0a6a-44b1-acb2-76ddbf521ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.399668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4890e7-0a6a-44b1-acb2-76ddbf521ef4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.399668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/851] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:54.401486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.401771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.402048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.421759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:54.423115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:54.436503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.452831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:54.454288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.517281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/852] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:54.546779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.593223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 420/853] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:50:54.599721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.615961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.671042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37089aca-de3e-4f0f-86a0-17400f137062 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.708897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.709453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.709741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.710179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.720797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:54.725504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:54.726741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:54.727890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/854] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:54.733709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.743673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d729d-5b52-400a-9163-5618cc335f9b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.746513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.792975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.794086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.794371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.794648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.810521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:54.812934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:54.834112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:54.835143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:54.836505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:54.836505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:54.836505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:54.836505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 421/855] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:54.847666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.869627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50c551-f305-4a27-864d-9630581cb96d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:54.910490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:54.911100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:54.911378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:54.911682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:54.912612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:54.912612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:54.914606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:54.924001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:54.926335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/856] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:54.931251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:54.975057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6ad98-56c3-48db-8942-967eefa46d33 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.012396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:55.016818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.017447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.017728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.018004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.018660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:55.018963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.020358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.038590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 422/857] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:55.039618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.049385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9f21-85b9-4a5e-8450-96b07aee3ebc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.084182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.084949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.085324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.085688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.097010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.098586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:55.112826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:55.114013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:55.115115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:55.115115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:55.115115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:55.116474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/858] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:55.122393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.154672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cbf85-6f78-480e-be3b-96ebff206a05 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.188791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:55.204096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.205049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.205333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.205606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.214828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.216571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:55.231308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:55.232452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:55.233583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:55.235678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 423/859] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:55.238853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.264207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9ee81-9238-4cbb-9636-8251b16b7672 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.290522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:55.310870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.311487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.311958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.312344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.312993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:55.313301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.315550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.337195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/860] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:55.338600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.371171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c0c8a-c17f-4ee2-9c74-abe762f745ae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.408010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.408933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.409406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.409930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.410846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:55.411456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.413106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.430707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 424/861] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:55.432315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.473989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36ba7-8d33-4eaa-9abc-2d207ee0b251 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.485374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:55.526016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.527304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.527584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.527885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.537789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.539744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.553239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa2e9-502d-4d58-bdad-2a3626126709 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:50:55.554492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:55.555402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:50:55.556677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:55.559024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/862] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:55.565452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.597565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.598179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.598482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.598991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.600148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=8f3a4e99f3044d35aa2e310478f3653b) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:55.600484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.602227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.618992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 425/863] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/role_assignments?user.id=8f3a4e99f3044d35aa2e310478f3653b&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:55.619384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:55.690460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612debe-0e9e-4b61-a9b6-5060b6ddb632 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.728759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.729942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.730455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.730741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.731490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:55.731832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.733302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2d121-24c8-4288-8504-e302cc9478b9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:55.755881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/864] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:55.764751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:55.879999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dae53-d98c-40e0-b2c1-d3b3d40db76a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:55.890602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:55.917681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:55.918381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:55.918787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:55.919129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:55.992362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:55.993911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.103856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 426/865] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:50:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:50:56.109540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38bb75-65c2-4a6e-98b0-f999efd4e9b2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.110005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38bb75-65c2-4a6e-98b0-f999efd4e9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.110099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38bb75-65c2-4a6e-98b0-f999efd4e9b2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.111298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:56.111884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/866] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:56.117088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.117606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.118088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.122809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:56.144439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:56.146675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:56.222143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbde6d-ea83-47b0-bbb4-628f9e68be46 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:56.246314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/867] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:56.261199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:56.262103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.262593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.263073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.269775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:56.278393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:56.280084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:50:56.295997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:56.297180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:50:56.298283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:50:56.298283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:50:56.298283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:56.299971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 427/868] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:56.305476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:56.310832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:56.388599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c5fda-9e27-437a-8d8f-cb808917e497 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:56.389492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:56.435909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:56.437394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.437831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.438252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.439061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:56.439507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1e0c4-ff49-404c-9c25-1a09cee6ea71 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:56.440755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.460951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/869] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:50:56 2022] POST /identity/v3/services => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:50:56.490176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:56.491107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.491552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.491987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.492793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:56.493258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:56.494817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda84a-65bb-4a34-be7c-b0bcd3e3a3b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.513663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 428/870] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:56.516501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:56.533194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee287b-751a-4741-b632-bc8f2cfba08f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.533884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee287b-751a-4741-b632-bc8f2cfba08f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.534432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee287b-751a-4741-b632-bc8f2cfba08f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.538846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 429/871] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:50:56.542502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.543017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.543442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.571989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:56.574937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:56.626726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:56.644880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618b42-85ba-4e57-ac50-814f58e03a0f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:56.677792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 430/872] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:56 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:50:56.689794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:56.690727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.691234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.691750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.700876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:56.706128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:56.707885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:50:56.723498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:56.724449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:50:56.725325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:56.728509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/873] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:56.734940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:56.758828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:56.815922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a828-4c9e-4185-a15d-8200f6f8672d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:56.861279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:56.861846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.862131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.862859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.863832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:56.865491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.871798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9402-5d01-4a38-badd-36c4e4cc3c51 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:56.888926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 431/874] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:50:56 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:50:56.928245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:56.928782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:56.929072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:56.929349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:56.930053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:56.930359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:56.932033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90e496-e506-4d7a-a69b-9c490c12f946 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:56.951762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/875] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:56.956259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:57.084371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a502d-dae3-41e6-8ed1-9c3c9d2e4f01 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:57.092926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:57.122204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:57.122814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:57.123096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:57.123374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:57.124134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=4dd6c53afc054e2384dc0d247a9b935c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:57.124442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:57.125869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4013473-a935-4a4c-a02f-397c0ef0999e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:57.132819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 432/876] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:50:56 2022] GET /identity/v3/role_assignments?user.id=4dd6c53afc054e2384dc0d247a9b935c&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:57.832256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da1b3c-ad86-4a46-8de0-ca32a7cfac9b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:57.832774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da1b3c-ad86-4a46-8de0-ca32a7cfac9b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:57.832774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da1b3c-ad86-4a46-8de0-ca32a7cfac9b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:57.834356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/877] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:57.843761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:57.844051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:57.844333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:57.868010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:57.869202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:57.939374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 433/878] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:57.961542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da1b3c-ad86-4a46-8de0-ca32a7cfac9b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:57.966241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:58.077369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da1b3c-ad86-4a46-8de0-ca32a7cfac9b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:58.113244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:58.113808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:58.114091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:58.114370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:58.114959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:58.116352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:58.125690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:58.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:58.127846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/879] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:50:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:58.132855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:58.239065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610dd99-bf46-41c4-9447-c54039820651 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:58.248736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:58.276657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:58.277611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:58.278099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:58.278579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:58.279446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:58.279982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:58.281558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44211ed-87c6-43e3-8eaf-df5958656e60 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:58.286909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 434/880] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:50:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:58.292403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:58.362657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:58.399298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a6b08-9c6e-4464-aba8-0dc1d8ba9c9b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:58.435356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:58.435990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:58.436268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:58.436540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:58.437168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:58.437469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:58.438817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f529ae-5821-4f95-a193-e2fdb9396242 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:58.447699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/881] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:50:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:58.644175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670c131-9ed4-4748-8634-50f7dbde1a7c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:58.644549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670c131-9ed4-4748-8634-50f7dbde1a7c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:58.644746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670c131-9ed4-4748-8634-50f7dbde1a7c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:58.646363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 435/882] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:58.652238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:58.652472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:58.652741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:58.671801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:58.673058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:58.742887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/883] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:58.766859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670c131-9ed4-4748-8634-50f7dbde1a7c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:58.883271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670c131-9ed4-4748-8634-50f7dbde1a7c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:58.920333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:58.920953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:58.921233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:58.921507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:58.922145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:58.923504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6f72907d-932a-4238-99ff-b1938df3a0a7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:58.933077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:58.934049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:58.935080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 436/884] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 9 06:50:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:58.942332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.007625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.039478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbfd5b-7be3-41b7-9005-a88c145a0c93 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.040494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbfd5b-7be3-41b7-9005-a88c145a0c93 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.040967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbfd5b-7be3-41b7-9005-a88c145a0c93 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.043075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 437/885] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:59.048233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.048497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761c679-85cc-43a2-b2c3-2ad7924341b9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.048989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.049474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.069144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:59.070753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:59.085284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.085907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.086195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.086474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.087110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:59.087417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.089465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.100890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/886] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:50:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:59.102107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.103118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.173562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 438/887] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:50:59.196023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.231396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf19f5-105b-4e29-955f-2a6053f30e67 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.267185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.282321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.282939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.283223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.283501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.284183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:59.284488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.285851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fbe1-ed8a-4a47-a630-6a3633d993c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.295327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/888] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:59.295327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89f446-71f4-49ae-8e05-2557d0e85b5f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.295327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89f446-71f4-49ae-8e05-2557d0e85b5f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.295327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89f446-71f4-49ae-8e05-2557d0e85b5f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.295327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/889] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:50:59.296875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89f446-71f4-49ae-8e05-2557d0e85b5f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.317326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb45079-317d-468f-b803-370252e7f8e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.354552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.355177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.355462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.355750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.356328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.357681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-749bb2a7-acbf-4fbb-ae1f-29c466bc914e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:59.366892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:59.367898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:50:59.368762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:50:59.373405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 439/890] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:50:59.374460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.374742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.375013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.393924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:50:59.395100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:50:59.406447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89f446-71f4-49ae-8e05-2557d0e85b5f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.413050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.442541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.443083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.443366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.443668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.444368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] RBAC: Adding query filter params (interface=public, service_id=7ec192a69dc84399ac29e38353a7d60f, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:59.444678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.446041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.458507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/891] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/endpoints?service_id=7ec192a69dc84399ac29e38353a7d60f&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:50:59.458507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.474116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 440/892] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:50:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:50:59.475186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.527055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.574529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da013a8-9c00-4431-8c59-17ced1014790 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.594268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71877-80ef-46aa-a99a-c312e2ca8744 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.609382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.609995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.610277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.610551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.611189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:59.611492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.612906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.623333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/893] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:59.623333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.640238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.640867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.641130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.641406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.642025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.643396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.659630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 441/894] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:50:59.659630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.679344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.765250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbc2e9-57a4-4ebd-a9a8-1e869eb0e3b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.771286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fc0db-7eae-4b34-aa29-6951f3d191bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.787470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.810689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.811484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.811923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.812271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.814517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:50:59.815469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:50:59.815787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:50:59.816064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:50:59.816727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:50:59.817033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:50:59.818416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:50:59.824419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 442/895] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:50:59.826170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:50:59.831648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:50:59.832613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:50:59.833543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:50:59.835630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.837919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/896] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:50:59.840259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:50:59.964397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:50:59.972229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f3299-28c5-4505-a4eb-8b116151a561 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:50:59.976383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be651227-368b-4f22-8775-d6079030bba8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.010141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.011627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.011955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.012331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.013183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:00.013613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.015069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.025926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.027422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.027810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/897] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:00.028014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.028414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.029659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] RBAC: Adding query filter params (interface=public, service_id=4fe3e7dc00de41ce8b35eaecd6aed0f5, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:00.030145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.030564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.031968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.044366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 443/898] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:50:59 2022] GET /identity/v3/endpoints?service_id=4fe3e7dc00de41ce8b35eaecd6aed0f5&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:00.046899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.079644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:00.142881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299784c-2811-4802-878c-e03f542f2b4c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.179479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae50ef-7c7d-4f32-b756-ce0ea450692d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.183300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.184226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.184717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.185215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.199927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.202244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.216909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.217531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.217806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.218076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.218687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.220159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:00.223320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:00.224289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:00.225176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:00.225176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:00.225176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:00.227764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/899] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:00.230962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.239670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 444/900] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:51:00 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:51:00.342338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:00.352828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e6af1-2a2a-4b68-8589-385d16dbf533 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.390408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.391032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.391392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.391695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.392355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:00.392659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.394044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1283fd8-cc92-422b-b60a-5a06556db2fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.408423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/901] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:00.414183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.427272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:00.530482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b68aa8-f2f4-4156-988f-54966d5d366d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.542000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d417a-b366-4752-853c-2c1bf2a3722f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.542355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d417a-b366-4752-853c-2c1bf2a3722f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.542637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d417a-b366-4752-853c-2c1bf2a3722f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.544690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/902] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:00.549923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.550172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.550446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.569689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:00.570992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:00.576709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.577630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.578106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.578571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.586815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.588278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:00.601145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:00.602089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:51:00.603056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:00.607305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 445/903] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:51:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:00.608285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.648943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/904] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:00.674159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.686952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:00.725306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05be6d5-dac1-4f1a-a4c8-0e2553bbce7f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.765925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.766615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.766873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.767216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.767902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:00.768205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.769638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.785942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:00.792039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 446/905] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 9 06:51:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:00.796220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:00.808487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52506e-8dda-41c7-b16e-4e970234be2c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.846944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.847916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.848393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.848862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:00.849711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:00.851332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:00.873220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/906] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:51:00 2022] POST /identity/v3/services => generated 241 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:51:00.918054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130df89-0d37-4124-a270-47e144beffbc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:00.950780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:00.965743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:00.966421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:00.966752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:00.967025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.042656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:01.044130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:01.148021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 447/907] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 06:51:01.155006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:01.263756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32ef22-6a8b-4028-8c02-0b351fd897a8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:01.300638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:01.301634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:01.302107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:01.302585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.314100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:01.315880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:01.317779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:01.332131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:01.333134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:01.334009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:01.334009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:01.334009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:01.336133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/908] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:01.342913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:01.369205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:01.451476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08298a5d-7d29-4dba-ae27-f872b29801d4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:01.490183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:01.491205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:01.491738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:01.492228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.493106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:01.493610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:01.495412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:01.508830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 448/909] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:01.515462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:01.621406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c65aa4-5dd6-41c1-9695-58adeb67d466 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:01.657546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:01.658556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:01.659030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:01.659515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.666578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:01.668660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-712600de-d883-459b-bb95-3b1fd7ef3a42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:01.680209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:01.681183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:01.682059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:01.684874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/910] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:01.690746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:01.778085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:01.796968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffeea3-9e9e-4371-8132-f7a2e37793e0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:01.832784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:01.833393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:01.833672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:01.833943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.834637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:01.834939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:01.836386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:01.847897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1384aa-298b-422a-b5f7-711a6d129eb3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:01.848618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1384aa-298b-422a-b5f7-711a6d129eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:01.848893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1384aa-298b-422a-b5f7-711a6d129eb3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.850761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/911] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:01.852124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 449/912] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:01.861220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:01.861566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:01.861850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:01.862118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:01.885157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:01.887661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:01.952084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:01.975645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/913] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:01.980680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e251-77b4-41e5-95d0-cd870581314b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:01.998963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:02.021761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.023140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.023623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.024342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.025249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] RBAC: Adding query filter params (role.id=eb9f9897e742453580668a6cbb647017, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=4dd6c53afc054e2384dc0d247a9b935c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:02.025828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.028398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.037084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 450/914] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:01 2022] GET /identity/v3/role_assignments?user.id=4dd6c53afc054e2384dc0d247a9b935c&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=eb9f9897e742453580668a6cbb647017 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:02.118217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a12592-2fed-43a5-ab83-a28c62f2cc5a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:02.157463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.158820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.159271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.159738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.160476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.162027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:02.173072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:02.174241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 06:51:02.175280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:02.177743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/915] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 9 06:51:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:02.185101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:02.201588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:02.301313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820d825-2e49-4518-b272-4f4eb482ac10 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:02.336263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.336916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.337201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.337518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.338156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:02.339040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.340860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa27f3b-4ada-42b8-b6a7-e17b27de16b9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.346201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 451/916] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:51:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:02.352884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:02.407842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:02.431185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d0db4-7cd1-4041-9bb7-b33b94108930 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.431494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d0db4-7cd1-4041-9bb7-b33b94108930 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.431774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d0db4-7cd1-4041-9bb7-b33b94108930 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.433846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 452/917] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:02.439276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.439906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.440394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.460116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:02.461699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:02.467216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745ee3-cd03-47d0-b2a3-1fa240f9f4fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:02.512262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.513276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.513566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.513969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.514569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:02.514875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.516705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.523968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/918] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:51:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:02.529184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:02.537907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:02.551693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 453/919] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:02.570370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:02.662925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6017801-6213-4f70-83ff-8ab4022c097c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:02.682879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:02.702650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a386d97-e27b-4c10-a687-dc672c472454 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:02.711194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.712528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.713055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.714161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.715393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.717594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.743931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/920] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:51:02 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:51:02.762596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.763254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.763534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.763847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.764508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.767440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:02.779080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:02.780212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:02.781262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 454/921] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:51:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:02.788698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:02.922272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae806f-4751-4c69-bd25-571d39179278 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:02.960875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:02.961585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:02.961937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:02.962280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:02.963079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:02.963572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:02.965327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15beb5-c18e-4a06-9265-3a259ad392c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:02.970814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/922] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:51:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:02.977246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.010625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee618a3-dc5c-4be0-a1cd-782f2314a340 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.011035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee618a3-dc5c-4be0-a1cd-782f2314a340 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.012306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee618a3-dc5c-4be0-a1cd-782f2314a340 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.012899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/923] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:03.021840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.022357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.022836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.042307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:03.044297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:03.048288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:03.079576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:03.097719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f978-6868-49fd-9fb0-3e19329f11fd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.135791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.136325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.136601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.136873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.137504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:03.137801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.139156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:03.139701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/924] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:03.144591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 455/925] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 9 06:51:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:03.152907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.161101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.277771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7a0a8-1c8d-4e89-b1f8-e24d9af682af None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.291034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04ee09-c236-4f34-9ff1-8019a633d428 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.303320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:03.314877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.315655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.316034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.316406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.317085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.318550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:03.327979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:03.329188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:03.330228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:03.336501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 456/926] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:51:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:03.337360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.338146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.339560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.340186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.340656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.341582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.343186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:03.371677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/927] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:51:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:03.461603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f15b99-2f9c-46c7-a6b8-2f689a507eeb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.507349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.508398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.508876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.509344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.510252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:03.510774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.512389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:03.518553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 457/928] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:03.524056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.546888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:03.636944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b47a2-7df6-4cab-abb7-a033e78b1317 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.671117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.671992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.672412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.672821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.673617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:03.674069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.675564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:03.680947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/929] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:03.687355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.739841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:03.794402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa7310-ed9d-4440-9a3d-0f7741c1d198 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.829287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.830134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.830559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.830991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.831846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] RBAC: Adding query filter params (interface=public, service_id=5dfce61e7b45477aa36b2972fa0778ce, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:03.832291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.833781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca27d67-49d0-4767-8389-752d33a329a0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:03.840472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 458/930] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:51:03 2022] GET /identity/v3/endpoints?service_id=5dfce61e7b45477aa36b2972fa0778ce&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:03.846086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:03.934891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:03.954733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e8f95-fc14-421e-a982-dc91b2fbfce0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:03.991046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62063a1b-4022-4f09-998d-6451ca3b920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:03.991583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62063a1b-4022-4f09-998d-6451ca3b920e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:03.991882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62063a1b-4022-4f09-998d-6451ca3b920e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:03.992157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62063a1b-4022-4f09-998d-6451ca3b920e None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:03.992840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62063a1b-4022-4f09-998d-6451ca3b920e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:03.994205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62063a1b-4022-4f09-998d-6451ca3b920e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:04.001046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/931] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:51:03 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:51:04.107568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276049-9d11-46c9-acda-59632b112ef2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.107955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276049-9d11-46c9-acda-59632b112ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.108137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276049-9d11-46c9-acda-59632b112ef2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.109722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 459/932] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:04.116628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.116914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.117187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.137003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:04.138214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:04.208456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/933] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:04.232142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276049-9d11-46c9-acda-59632b112ef2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:04.348666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276049-9d11-46c9-acda-59632b112ef2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:04.362836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:04.387248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:04.387985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.388305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.388583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.399896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:04.405170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:04.406101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:04.407066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 460/934] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 9 06:51:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:04.412203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:04.495667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:04.522711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54427d2-3e55-4778-9cc6-cfa7e334c752 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:04.559790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:04.560706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.561261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.561718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.562568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:04.563051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:04.564624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:04.574806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/935] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:51:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:04.580135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:04.691845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb572cfa-6913-483d-95da-f87ac11c595b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:04.727982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:04.728618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.728926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.729204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.740501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:04.741951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:04.756710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:04.757667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:04.759014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:04.759014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:04.759014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:04.759014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 461/936] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:04.764763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:04.885061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983e7a9-1128-4af8-9f02-43b609b20acf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:04.920792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:04.921402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.921684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.921961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.922605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:04.922911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:04.924314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a9fe7-1841-4d77-a20f-793e68d98522 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:04.931495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:04.939468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/937] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:04.949439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:04.954628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e3a53-83d3-49b3-a465-ea6203e4d1fd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.955144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e3a53-83d3-49b3-a465-ea6203e4d1fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.955685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e3a53-83d3-49b3-a465-ea6203e4d1fd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.957421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/938] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:04.962553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:04.963114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:04.963623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:04.978125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:04.983430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:04.984891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:05.063910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/939] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:05.076552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d143f5f-4e75-4b2b-a7e0-fea16f8998bb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.080900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.121074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148ce86-402c-494f-8372-89208190e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.121719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ce86-402c-494f-8372-89208190e08e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.122001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ce86-402c-494f-8372-89208190e08e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.122279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ce86-402c-494f-8372-89208190e08e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.132575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148ce86-402c-494f-8372-89208190e08e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.134335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148ce86-402c-494f-8372-89208190e08e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7148ce86-402c-494f-8372-89208190e08e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:05.148198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:05.149187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:51:05.150107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.157509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 462/940] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:51:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:05.158810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148ce86-402c-494f-8372-89208190e08e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.204429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f04c54-b6cf-4d7e-ad57-d4c864fd9d90 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.233894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:05.243466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.244190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.244414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.244986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.245580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.246950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.256004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:05.256305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:05.257246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:05.258057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:05.262970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/941] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:05.264428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.272274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148ce86-402c-494f-8372-89208190e08e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.308787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.309819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.310309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.310862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.311807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:05.312306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.314040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.339260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 463/942] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:05.340529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.370515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8665ec7-71b8-405f-90ee-8274dfdf9672 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.406701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.407323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.407802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.407920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.408516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:05.408886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.410171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.419906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/943] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:05.420784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.448786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f97d9-c3df-49c4-8452-04d6092459b4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.480196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:05.488508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.489407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.489908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.490381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.502663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.504589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.506612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:05.522426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:05.523403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:05.524328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:05.524328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:05.524328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.531283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d4785-23d9-469e-ac05-3d28ba1f906e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.531702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 464/944] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:05.532854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.565312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.565988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.566274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.566620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.567262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:05.567563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.568995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664830-b1a0-4e1d-bcba-d02fef6b9a4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.574792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/945] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:05.585628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc3f22-7059-450c-94bd-cea61e17bbbb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.586011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc3f22-7059-450c-94bd-cea61e17bbbb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.586191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc3f22-7059-450c-94bd-cea61e17bbbb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.592785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/946] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:05.594211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.594493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.594762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.614756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:05.616011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:05.655815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51805-e7f0-4d6d-9a6f-c120e38f0f9d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.704548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.705593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.706069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.706537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.707394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:05.707931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.709549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.711747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/947] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:05.726532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 465/948] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:05.732687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.737372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.789842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:05.826920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:05.848332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf7392-bd93-496c-9150-3845d21979e9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.885010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.886309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.886668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.886912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.889416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c42a5-1057-4267-8e3a-4ee0f5f8e3b6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:05.894316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.895927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:05.908429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:05.910336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:05.913154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:05.919620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/949] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:05.919620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:05.929630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:05.929940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:05.930149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:05.930424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:05.943643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:05.943643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:05.943643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING keystone.server.flask.common [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 06:51:05.957501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 06:51:05.957501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 06:51:05.957501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [parameters: {'id': 'c01146ddd7534c66b26f6f9187ca37e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 06:51:05.957501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c01146ddd7534c66b26f6f9187ca37e6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c01146ddd7534c66b26f6f9187ca37e6. Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 06:51:05.972137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 06:51:05.974005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() Aug 09 06:51:05.977340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 06:51:05.980391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 06:51:05.982548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 06:51:05.992777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': 'c01146ddd7534c66b26f6f9187ca37e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.995524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:05.997713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c01146ddd7534c66b26f6f9187ca37e6. Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 466/950] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Aug 9 06:51:05 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:06.000502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:06.034910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032a5ea-e594-4480-b844-21f3db1db0d1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.047102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:06.074870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.075560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.075880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.076210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.077046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:06.077356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.080236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.098209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/951] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:06.099326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:06.117136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b268ac-7a07-42ff-9357-6a71bfea1fe3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.129515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:06.165393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.166032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.166955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.167453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.178285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.179976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f707937d-5c8b-46c7-b833-a4381453f4a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:06.197903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:06.198905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:06.200023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 467/952] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:06.206006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428df13-0e4e-4ab7-a44e-c976056b2c75 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.206278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428df13-0e4e-4ab7-a44e-c976056b2c75 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.206544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428df13-0e4e-4ab7-a44e-c976056b2c75 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.215570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 468/953] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:06.215570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428df13-0e4e-4ab7-a44e-c976056b2c75 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:06.227725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfb702-79f0-4fc9-a1df-13e5d7986d13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.273993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93ed99-177b-4cfc-abd7-a272f04761a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.275048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93ed99-177b-4cfc-abd7-a272f04761a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.275529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93ed99-177b-4cfc-abd7-a272f04761a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.276570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93ed99-177b-4cfc-abd7-a272f04761a1 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/4dd6c53afc054e2384dc0d247a9b935c/roles/eb9f9897e742453580668a6cbb647017` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.279022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93ed99-177b-4cfc-abd7-a272f04761a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=4dd6c53afc054e2384dc0d247a9b935c, role_id=eb9f9897e742453580668a6cbb647017)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.280485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93ed99-177b-4cfc-abd7-a272f04761a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.296491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/954] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:51:06 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/4dd6c53afc054e2384dc0d247a9b935c/roles/eb9f9897e742453580668a6cbb647017 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:51:06.297745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.298086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.298352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.319000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:06.320305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:06.353606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:06.361490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428df13-0e4e-4ab7-a44e-c976056b2c75 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.412222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.413092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.413553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.414001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.415223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:06.415532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.417205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.434432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/955] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:06.445085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 469/956] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:06.455936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:06.585411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50941985-e287-4adf-a603-1f910c67f0ef None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.627412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.628432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.628839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.629254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.629984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.631583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:06.647229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:06.648448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:06.649407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/957] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:51:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:06.654704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:06.691417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:06.780140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d65b0-9ee1-4526-be29-48b1a4bc5b0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.815583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.816549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.816993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.817422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.818201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:06.818651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.820261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.830372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 470/958] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:06.832588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:06.867185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:06.948284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb66f0c-9178-4fe1-a80a-a1c6940174a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:06.984888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:06.985504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:06.985779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:06.986050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:06.986677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:06.986974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:06.988375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710463f5-bbc0-4873-bc93-50a1de94a852 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:06.993812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/959] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:06.999881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:07.098546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:07.105771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0d4a8-2808-471f-b6aa-830ce6b6d3eb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:07.140775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df0978-22ed-4d2a-9304-10ec1912172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:07.141401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df0978-22ed-4d2a-9304-10ec1912172f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:07.141681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df0978-22ed-4d2a-9304-10ec1912172f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:07.141953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df0978-22ed-4d2a-9304-10ec1912172f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:07.142623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df0978-22ed-4d2a-9304-10ec1912172f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:07.144122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df0978-22ed-4d2a-9304-10ec1912172f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:07.171744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 471/960] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:51:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:07.856934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec35f2e-380d-4ba3-8cc5-6eeccb037d75 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:07.857434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec35f2e-380d-4ba3-8cc5-6eeccb037d75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:07.857434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec35f2e-380d-4ba3-8cc5-6eeccb037d75 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:07.859066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/961] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:07.864266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89829dde-d1aa-44a4-ac67-52a39376db9f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:07.864656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89829dde-d1aa-44a4-ac67-52a39376db9f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:07.864863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89829dde-d1aa-44a4-ac67-52a39376db9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:07.883472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-89829dde-d1aa-44a4-ac67-52a39376db9f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:07.885100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-89829dde-d1aa-44a4-ac67-52a39376db9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:07.958909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 472/962] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:07.981639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec35f2e-380d-4ba3-8cc5-6eeccb037d75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:08.072678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:08.086747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec35f2e-380d-4ba3-8cc5-6eeccb037d75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:08.121975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a68d32-8c04-4ec0-bb8d-98e5d86970d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:08.122498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a68d32-8c04-4ec0-bb8d-98e5d86970d1 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.122776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a68d32-8c04-4ec0-bb8d-98e5d86970d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.123047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a68d32-8c04-4ec0-bb8d-98e5d86970d1 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.123724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a68d32-8c04-4ec0-bb8d-98e5d86970d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:08.125136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a68d32-8c04-4ec0-bb8d-98e5d86970d1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:08.143364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/963] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:51:07 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:08.540433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b80be-9a99-4d58-af2e-6f2f85c7603b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.541160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b80be-9a99-4d58-af2e-6f2f85c7603b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.541784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b80be-9a99-4d58-af2e-6f2f85c7603b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.543637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 473/964] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:08.548942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623b92d-3cbb-4bc2-b30a-4a4e5c0ffcec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.549463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623b92d-3cbb-4bc2-b30a-4a4e5c0ffcec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.549952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623b92d-3cbb-4bc2-b30a-4a4e5c0ffcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.568532 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a623b92d-3cbb-4bc2-b30a-4a4e5c0ffcec None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:08.570081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a623b92d-3cbb-4bc2-b30a-4a4e5c0ffcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:08.654085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/965] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:08.676788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b80be-9a99-4d58-af2e-6f2f85c7603b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:08.679206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ae0b5-f10b-49f0-8a53-79fbff334acb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.679496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ae0b5-f10b-49f0-8a53-79fbff334acb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.679798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ae0b5-f10b-49f0-8a53-79fbff334acb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.681127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/966] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:08.686835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.687120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.687398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.711191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:08.712817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:08.776580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:08.783716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/967] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:08.804256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b80be-9a99-4d58-af2e-6f2f85c7603b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:08.808552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:08.813186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:08.848216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:08.848798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.849078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.849358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.866146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:08.871814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:08.872758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:08.873588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:08.878219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 474/968] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 9 06:51:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:08.879475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:08.944562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c3a01-a90c-4ab7-adc1-da813e4d6ac3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:08.984325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98b93c-fb51-4959-bc7e-3f429876d4df None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:08.990656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:08.991453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:08.991921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:08.992729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:08.993516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:08.995471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:08.998958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING keystone.server.flask.common [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 06:51:09.016610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 06:51:09.016610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 06:51:09.016610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [parameters: {'user_id': '15d8deea1eb942dc90293bff2648282d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 06:51:09.016610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 06:51:09.021579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.022369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.022786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.023166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.024037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:09.024539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.026658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.035932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Aug 09 06:51:09.038159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 06:51:09.039072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 06:51:09.040030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 06:51:09.040947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 06:51:09.042158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 06:51:09.043227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'user_id': '15d8deea1eb942dc90293bff2648282d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:09.044303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 06:51:09.045112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 06:51:09.046064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 06:51:09.046064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 06:51:09.046064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 06:51:09.046064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 06:51:09.046064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 06:51:09.046064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.046550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 475/969] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:51:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:09.047851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.048520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/970] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 9 06:51:08 2022] POST /identity/v3/users => generated 160 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:09.054166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.143127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.164068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5b1f1-b412-4a6e-93bc-eae342382528 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.183398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194a16-d3ac-4594-a79c-2f4565863627 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.201196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.201988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.202458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.202844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.214836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.216488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.222355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.223170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.223650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.224135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f6f95cb8-d563-4d3b-9e9b-9f6d15e34ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:09.232107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:09.233065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:09.233944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:09.233944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:09.233944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:09.234115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.234115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.237979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 476/971] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:09.239226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3275-3dc0-4f9b-b48e-ebd534edda54 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.239515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3275-3dc0-4f9b-b48e-ebd534edda54 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.239817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3275-3dc0-4f9b-b48e-ebd534edda54 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.241166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 477/972] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:09.245073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d3275-3dc0-4f9b-b48e-ebd534edda54 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-012b9b79-74e8-4a0a-ac15-8a66ae0b165f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:09.246780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:09.247768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.248686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/973] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:09.250623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.250623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.250623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.268910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:09.270424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:09.302508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.345455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/974] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:09.346310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.374431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d3275-3dc0-4f9b-b48e-ebd534edda54 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.405327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.415053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.415692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.415983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.416297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.416942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:09.417248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.418619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.431202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 478/975] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:09.432890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.457336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fc826-25de-4105-8614-2e6adc7b93d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.496409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.497234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.497838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.498117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.498971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:09.499272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.500716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.540342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/976] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:09.541208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.541887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08524a05-8612-4d85-bd03-f42c6ecefe9e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.561294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.601357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.602177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.603011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.603391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.605003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.607014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:09.620085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:09.621154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:09.622008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:09.623342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 479/977] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:09.628022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.687466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e30b2-90cb-436d-9415-8720ad3b969f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.706046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.733029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.733675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.733957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.734231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.742373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.744176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.754934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66b25e-dfc7-4ec8-95e3-9e1465378690 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:09.760972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:09.762085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:09.763328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/978] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:09.769452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.799391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.800445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.800728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.801113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.801758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:09.802064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.803423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.810903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 480/979] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:09.816638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.865508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:09.903234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c560b90-fc8c-406c-84fe-e10a25c980a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.929406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620323f-8582-4a87-8d2f-56d440abbeb5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:09.943916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.944540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.944897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.945431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.945898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:09.946206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.947690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.965791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:09.970616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/980] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:09.970759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:09.970759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:09.970759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:09.970759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:09.970759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:09.970759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:09.971716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:09.975636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 481/981] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:09.982106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.006429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.081923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283713b-ad7e-4538-a434-c804a2b8ed0e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.093088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ce1f-e54f-4cc0-b717-f22020b67b8e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.095133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.119515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.120648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.120928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.121195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.131296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.131940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.132224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.132499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.133205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] RBAC: Adding query filter params (interface=admin, service_id=5dfce61e7b45477aa36b2972fa0778ce, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:10.133515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.134873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.145338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 482/982] 10.222.0.30 () {56 vars in 1249 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/endpoints?service_id=5dfce61e7b45477aa36b2972fa0778ce&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:10.146302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.194779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.196189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.230197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.254132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416ac36-5dba-4802-86f2-9d256d3b9bb7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.289114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43731d-5962-4edc-830c-aea261dd2ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.289727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43731d-5962-4edc-830c-aea261dd2ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.290011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43731d-5962-4edc-830c-aea261dd2ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.290292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43731d-5962-4edc-830c-aea261dd2ae7 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.290897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43731d-5962-4edc-830c-aea261dd2ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.292403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43731d-5962-4edc-830c-aea261dd2ae7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.298954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 483/983] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:51:10.300236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a147eb-b71a-4a9a-b303-9ce864384861 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.300528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a147eb-b71a-4a9a-b303-9ce864384861 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.300805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a147eb-b71a-4a9a-b303-9ce864384861 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.302208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 484/984] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:10.308329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.308618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.308897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.315125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/985] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:09 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:10.321091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.329642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:10.330894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:10.358119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.409459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 485/986] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:10.431338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.432958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f381c2-fd04-475b-b979-4911542c0e80 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.468132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.475528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.476279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.476564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.476839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.488502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.489945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:10.505994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:10.507159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:10.508257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:10.508257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:10.508257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:10.508257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/987] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:10.513846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.554009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbbd18-2265-422e-8725-8c88eca2f703 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.589003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.589652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.589934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.590207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.590809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.592215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:10.602293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:10.603289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:10.605478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 486/988] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:10.610169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.629872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8879ff4-50bd-4b76-92a7-ef23ac208ecd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.637512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.668125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.668937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.669343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.669618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.670259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:10.670562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.672080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.689904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/989] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:10.691215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.718699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f793f-8813-4418-bb7e-865c8cbf9642 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.739143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.761419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.762027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.762310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.762597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.763226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:10.763527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.764939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.770001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 487/990] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:10.775502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.805142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5687518-491b-4659-aaa4-852f88b16668 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.847058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.847855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.848039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.848274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.857371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.857371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:10.870380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:10.871551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:10.872515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:10.873061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:10.874846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/991] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:10.880587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.887297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d108e1-d8df-437c-835b-7db7fd329a54 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.924212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:10.925079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:10.925652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:10.926114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:10.926927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:10.927396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:10.929049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:10.935363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 488/992] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:10.940653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:10.987799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bea9dd-e280-480f-811c-d14dd04a44d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:10.991116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:11.027505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:11.028373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:11.028813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:11.029246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:11.030164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:11.030607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:11.032368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:11.041033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9be95-446c-49bb-accc-4bf50e83d97b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:11.046240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/993] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:11.052167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:11.080472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:11.081296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:11.081717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:11.082181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:11.083026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] RBAC: Adding query filter params (interface=public, service_id=4141eebfc63248feab38d7d770ec7c40, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:11.083334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:11.084724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:11.095584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 489/994] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:51:10 2022] GET /identity/v3/endpoints?service_id=4141eebfc63248feab38d7d770ec7c40&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:11.096592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:11.116113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:11.160231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd181b-8647-490d-b761-b0460605e812 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:11.196371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:11.196968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:11.197248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:11.197690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:11.198631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] RBAC: Adding query filter params (role.id=eb9f9897e742453580668a6cbb647017, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=4dd6c53afc054e2384dc0d247a9b935c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:11.198936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:11.200380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becabe96-7cee-43d9-a2b4-9b9b76fdc9d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:11.204687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a04990-3318-4689-ad46-dd4645f07863 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:11.206116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/995] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:11 2022] GET /identity/v3/role_assignments?user.id=4dd6c53afc054e2384dc0d247a9b935c&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=eb9f9897e742453580668a6cbb647017 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:11.242803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac1494-6bda-42cb-9017-1b84aecd3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:11.243347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac1494-6bda-42cb-9017-1b84aecd3be2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:11.243667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac1494-6bda-42cb-9017-1b84aecd3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:11.244017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac1494-6bda-42cb-9017-1b84aecd3be2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:11.244691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac1494-6bda-42cb-9017-1b84aecd3be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:11.246097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac1494-6bda-42cb-9017-1b84aecd3be2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:11.253587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 490/996] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:51:11 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 06:51:12.337861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78a8a-00ec-4fb9-a201-271eb3ae9109 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:12.338400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78a8a-00ec-4fb9-a201-271eb3ae9109 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:12.338749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78a8a-00ec-4fb9-a201-271eb3ae9109 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:12.340877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/997] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:12.346194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:12.346735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:12.346735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:12.366645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:12.367901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:12.438191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 491/998] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:12.460740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78a8a-00ec-4fb9-a201-271eb3ae9109 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:12.488269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:12.564021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78a8a-00ec-4fb9-a201-271eb3ae9109 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:12.597820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:12.598380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:12.598663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:12.598936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:12.599513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:12.600899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:12.610391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:12.611380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 06:51:12.612259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:12.615933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/999] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 9 06:51:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:12.617368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:12.629704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:12.718841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda502b7-5d07-4c5b-ab07-9d07e2e9f7cb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:12.753128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:12.753740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:12.754024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:12.754302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:12.754935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:12.755239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:12.756631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:12.766769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 492/1000] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:12.768065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:12.870635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:12.880903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c747a87-9ff2-47e1-b26d-6b82834ff42b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:12.918136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:12.918751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:12.919033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:12.919307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:12.919985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:12.920291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:12.921739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e472708-5e8a-4f10-9377-076f0404ce2a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:12.926956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/1001] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 9 06:51:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:12.933186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.013071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:13.039937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3e20-307f-4bc2-87a1-b26840526566 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.075748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618554bf-19a1-4694-86d2-088eb8d88d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.076357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618554bf-19a1-4694-86d2-088eb8d88d3f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.076635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618554bf-19a1-4694-86d2-088eb8d88d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.076907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618554bf-19a1-4694-86d2-088eb8d88d3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.077563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618554bf-19a1-4694-86d2-088eb8d88d3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.078924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618554bf-19a1-4694-86d2-088eb8d88d3f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.095080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 493/1002] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:51:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:13.262533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ae242-860a-42cd-8ca2-23a750e0ef69 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.262888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ae242-860a-42cd-8ca2-23a750e0ef69 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.263083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ae242-860a-42cd-8ca2-23a750e0ef69 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.264480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/1003] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:13.269991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.270287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.270569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.277204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbd24b-1192-4ed1-8a72-5b7ed89d4786 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.277488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbd24b-1192-4ed1-8a72-5b7ed89d4786 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.277762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbd24b-1192-4ed1-8a72-5b7ed89d4786 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.279242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/1004] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:13.284201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.284566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.284850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.289555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:13.290765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:13.303429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:13.304758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:13.364817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 494/1005] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:13.379638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/1006] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:13.388356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.403740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.448928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:13.493081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:13.496774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369436c-22df-414b-a0c8-b0f042825322 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.511117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557403f-7b83-4188-b3c4-2eb5820feb06 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.532029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.532665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.533004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.533341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.533986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.535420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:13.544808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:13.545831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:13.546694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 495/1007] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 9 06:51:13 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:13.550579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.551226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.551508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.551804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.551893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.552397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.553772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:13.564044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:13.568596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:13.569426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/1008] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:13.571140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.664002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637996ed-bf41-4f79-ad6f-e4ff9e519b42 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.674381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779b3e-e75c-4b68-b6b6-023c2b743dca None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.689503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:13.702434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.703044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.703325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.703624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.704258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:13.704562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.705933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.709688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.710300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.710578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.711075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 496/1009] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:13.715087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.715087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:13.715087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.715087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.716953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.724568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1010] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:13.726052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.742499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:13.822976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9b6c4-e219-43f3-bac8-bd68105ae737 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.834020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc0630-cbb8-473c-8e8e-9f53509f43c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:13.868499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.869961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.870249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.870529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.871163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:13.871466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.872860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.873273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:13.873995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:13.874275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:13.874545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:13.875168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:13.875465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:13.876829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191e6a1-e521-46f3-817c-dd3136c87f35 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:13.878259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 497/1011] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:13.887658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1012] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 9 06:51:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:13.889665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:13.949751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:14.001680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915627e-791b-4c0e-9d8f-dd2088876b59 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:14.040528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9f06a-f9c4-4dc4-a19f-601820d720a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:14.041531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9f06a-f9c4-4dc4-a19f-601820d720a3 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:14.042000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9f06a-f9c4-4dc4-a19f-601820d720a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:14.042269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9f06a-f9c4-4dc4-a19f-601820d720a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:14.042940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9f06a-f9c4-4dc4-a19f-601820d720a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:14.044334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9f06a-f9c4-4dc4-a19f-601820d720a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:14.066409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 498/1013] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:51:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:15.115071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a85f9-6e64-43cb-8ac5-3eecb57ae9c5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:15.115629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a85f9-6e64-43cb-8ac5-3eecb57ae9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:15.115629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a85f9-6e64-43cb-8ac5-3eecb57ae9c5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:15.117056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1014] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:15.122247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:15.122705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:15.122796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:15.142099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:15.143289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:15.223633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 499/1015] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:15 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:15.243145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a85f9-6e64-43cb-8ac5-3eecb57ae9c5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:15.326241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:15.350253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a85f9-6e64-43cb-8ac5-3eecb57ae9c5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:15.383525 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9fc32-fd8c-4ff1-acdb-1625eea255b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:15.384102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9fc32-fd8c-4ff1-acdb-1625eea255b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:15.384377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9fc32-fd8c-4ff1-acdb-1625eea255b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:15.384648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9fc32-fd8c-4ff1-acdb-1625eea255b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:15.386471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9fc32-fd8c-4ff1-acdb-1625eea255b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:15.387914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9fc32-fd8c-4ff1-acdb-1625eea255b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:15.389686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1016] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:15.395110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:15.428406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:15.496423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a83e-ed22-4dff-ad93-6a436b1af86a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:15.531322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b24cb-8a28-45a0-972e-aa54d470a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:15.531953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b24cb-8a28-45a0-972e-aa54d470a5a6 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:15.532227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b24cb-8a28-45a0-972e-aa54d470a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:15.532495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b24cb-8a28-45a0-972e-aa54d470a5a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:15.533136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b24cb-8a28-45a0-972e-aa54d470a5a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:15.534543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b24cb-8a28-45a0-972e-aa54d470a5a6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:15.566542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 500/1017] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:51:15 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:15.906475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3098c-499d-45c0-a571-2ff977f2b1a1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:15.906792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3098c-499d-45c0-a571-2ff977f2b1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:15.907042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3098c-499d-45c0-a571-2ff977f2b1a1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:15.908460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1018] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:15.913608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a677-dd35-4454-9e48-a1bb1ce283ae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:15.913899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a677-dd35-4454-9e48-a1bb1ce283ae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:15.914178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a677-dd35-4454-9e48-a1bb1ce283ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:15.932805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1a59a677-dd35-4454-9e48-a1bb1ce283ae None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:15.934011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a59a677-dd35-4454-9e48-a1bb1ce283ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:16.004460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 501/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:15 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:16.027804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3098c-499d-45c0-a571-2ff977f2b1a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:16.135969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3098c-499d-45c0-a571-2ff977f2b1a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:16.168958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a15e10-f610-4577-be42-780309155025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:16.169630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a15e10-f610-4577-be42-780309155025 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.169913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a15e10-f610-4577-be42-780309155025 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.170187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a15e10-f610-4577-be42-780309155025 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.171899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a15e10-f610-4577-be42-780309155025 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:16.172198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9d7c-166e-44fc-9f01-73669ce4ae24 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.172618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9d7c-166e-44fc-9f01-73669ce4ae24 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.172776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9d7c-166e-44fc-9f01-73669ce4ae24 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.173299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a15e10-f610-4577-be42-780309155025 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:16.174319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 502/1020] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:16.179188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.179488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.179852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.190617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1021] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:51:16 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:16.200031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:16.201262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:16.215870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:16.295629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 503/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:16.315577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a15e10-f610-4577-be42-780309155025 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:16.326338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:16.423426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a15e10-f610-4577-be42-780309155025 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:16.457530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:16.458163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.458444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.458718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.469746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:16.474740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:16.475739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:16.476631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1023] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 9 06:51:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:16.482802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:16.586630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad5d4-71ef-4f98-bb10-11399ad59b49 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:16.621024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:16.621690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.621978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.622251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.622917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:16.623223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:16.624621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:16.638475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 504/1024] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:51:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:16.639915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:16.707760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:16.746567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248283-f933-445c-aeef-6a77c0b6faab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:16.781934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:16.782569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.782848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.783117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.794978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:16.796434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:16.810468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:16.811528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:16.812888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:16.812888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:16.812888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:16.812888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1025] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:16.817210 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:16.820997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:16.931435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bf72f-7833-40cd-9f00-8ec69ac15e3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:16.965018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:16.965554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:16.965830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:16.966102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:16.966734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:16.967033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:16.968415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:16.980284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 505/1026] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:16.985598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.084468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73256bc3-c88e-455c-a57e-2aea1bd9951d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:17.101642 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:17.119858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:17.120482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.120758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.121027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.128855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:17.130275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:17.141252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:17.142619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:51:17.143589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:17.144915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1027] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:51:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:17.148060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.214998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:17.255503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c5a10-8fbc-472a-a7eb-d2144172070a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:17.289106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:17.289714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.289993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.290264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.291592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:17.291934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:17.293353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca62f08-cc10-4c6f-9aa3-3f218f924874 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:17.312601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 506/1028] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:51:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:17.317880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.422133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a820a-af45-4c3e-85fc-8b96b949ab46 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:17.456032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:17.456636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.456923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.457193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.503292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:17.526996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:17.528434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:17.623428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc478-27b1-4226-b436-fd88f70ff6da None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.623796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc478-27b1-4226-b436-fd88f70ff6da None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.624018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc478-27b1-4226-b436-fd88f70ff6da None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.625493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 507/1029] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:17.625934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1030] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:17 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:17.631052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.631336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.631522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.631668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.651479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:17.652703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:17.715291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:17.728758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 508/1031] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:17.742632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7236a6-d944-461e-8b2c-2e9b0ad708f5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:17.753241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.781769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:17.782336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.782628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.782893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.793704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:17.795148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:17.810874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:17.816764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:17.817718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:17.817718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:17.817718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:17.817718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1032] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:17.817718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.825890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:17.890241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6a792-d8eb-4b2c-a7d1-7fe17b0100fa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:17.928740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1527a4d-bd71-4ba1-9587-72a1b1d51299 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:17.929100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:17.929642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.929922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.930283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.932221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:17.933406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:17.939948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 509/1033] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:17.940632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:17.960707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:17.968131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:17.968804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:17.969030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:17.969307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:17.969947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:17.970251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:17.971627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:17.985085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1034] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:17.990376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:18.052650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df3bee-216d-4ad6-8f74-e9ea9ce450c7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.077788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:18.092823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3688-9572-4b25-beec-502482336126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.093525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3688-9572-4b25-beec-502482336126 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.093810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3688-9572-4b25-beec-502482336126 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.094083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3688-9572-4b25-beec-502482336126 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.094720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f3688-9572-4b25-beec-502482336126 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.096218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f3688-9572-4b25-beec-502482336126 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.102355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a132f5-fdc6-437e-a826-1648d869a127 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.119656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 510/1035] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:51:17 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:18.143043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.143790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.143993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.144260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.152132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.153596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9233a3eb-274d-4779-bbe0-caf3cf7ef412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:18.166628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:18.172723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:18.173975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1036] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:18.176439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3688-9572-4b25-beec-502482336126 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:18.258250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddab7f-150f-48ee-80f5-90a56e066571 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.258588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddab7f-150f-48ee-80f5-90a56e066571 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.258804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddab7f-150f-48ee-80f5-90a56e066571 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.260394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1037] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:18.265883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.266193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.266441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.285596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:18.287567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:18.295022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3688-9572-4b25-beec-502482336126 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.331922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.332460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.332738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.333010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.333705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:18.334040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.335432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.343438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:18.355432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 511/1038] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:18.361503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:18.374099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1039] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:18.396419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:18.398950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:18.468887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8c607-8bec-4c58-b2d7-c48d50ce4fae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.511530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.512172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.512281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1d99f-0a10-4cf3-abc7-6e09d0a2edca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.512462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.512722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.513449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] RBAC: Adding query filter params (role.id=eb9f9897e742453580668a6cbb647017, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=d80a908c1da74b3fb3e82c8277556e81) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:18.513751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.515188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.521843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 512/1040] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:18 2022] GET /identity/v3/role_assignments?user.id=d80a908c1da74b3fb3e82c8277556e81&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=eb9f9897e742453580668a6cbb647017 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:18.553924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.554870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.555326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.555799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.556622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.558523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:18.568736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:18.569945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:18.570806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:18.571653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1041] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 9 06:51:18 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:18.577583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:18.692581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd7806-7a78-495b-94a0-38064bc5d4d0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.694514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:18.730614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.731244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.731525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.731829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.732464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:18.732769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.734224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.743654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 513/1042] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:18.744950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:18.794877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:18.862278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b7bc-2f6f-48f1-9963-04bcb711de85 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:18.897486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:18.898165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:18.898452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:18.898728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:18.899365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:18.899718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:18.901202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:18.910881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1043] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:18.912213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:19.026991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689cd6-78ef-4551-a349-c073a297624d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:19.061218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:19.061825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:19.062107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:19.062385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:19.063100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] RBAC: Adding query filter params (interface=public, service_id=4c37f76b647e4c5a959028e0bf858e5e, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:19.063395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:19.064793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b2974-fcc2-4ae3-af0a-634f02b3fce5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:19.074545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 514/1044] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:51:18 2022] GET /identity/v3/endpoints?service_id=4c37f76b647e4c5a959028e0bf858e5e&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:19.075872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:19.180373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca1430-8f21-44ef-a813-0017787ff279 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:19.189576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:19.216289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f353e9-ec7d-40e6-b086-df0aa621e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:19.216906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f353e9-ec7d-40e6-b086-df0aa621e623 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:19.217191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f353e9-ec7d-40e6-b086-df0aa621e623 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:19.217466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f353e9-ec7d-40e6-b086-df0aa621e623 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:19.218075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f353e9-ec7d-40e6-b086-df0aa621e623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:19.219444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f353e9-ec7d-40e6-b086-df0aa621e623 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:19.231976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1045] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:51:19 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 06:51:20.194786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21387c-23ef-4f18-a2cc-81dad93ff148 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.195334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21387c-23ef-4f18-a2cc-81dad93ff148 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.195334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21387c-23ef-4f18-a2cc-81dad93ff148 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.196743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 515/1046] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:20.202046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.202465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.202659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.220932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:20.222104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:20.289686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1047] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:20.312022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21387c-23ef-4f18-a2cc-81dad93ff148 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:20.345361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:20.422177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21387c-23ef-4f18-a2cc-81dad93ff148 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:20.455832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bfccf-ce7a-4d6d-baa9-90b392d47b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:20.456387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bfccf-ce7a-4d6d-baa9-90b392d47b21 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.456664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bfccf-ce7a-4d6d-baa9-90b392d47b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.456933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bfccf-ce7a-4d6d-baa9-90b392d47b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.468097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bfccf-ce7a-4d6d-baa9-90b392d47b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-523bfccf-ce7a-4d6d-baa9-90b392d47b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:20.472774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:20.473681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:20.474539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:20.479056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 516/1048] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:51:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:20.480341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:20.517658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c87abe-456d-4692-8520-88938ff165da None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.518221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c87abe-456d-4692-8520-88938ff165da None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.518221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c87abe-456d-4692-8520-88938ff165da None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.523449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 517/1049] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:20.524600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.524920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.525744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.544433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:20.545614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:20.609439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec7bb2-1318-4ef8-8f97-d137ccbff8e4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:20.612563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:20.652938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:20.653796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.654214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 518/1050] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:20.654704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.655128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.655959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:20.656391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:20.657915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:20.667514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1051] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:20.672988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:20.675454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:20.696857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:20.794392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160f984-851f-4294-9f02-a85c295772b9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:20.798848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f8a6a-5892-4dfe-9403-9e11d724ff1e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:20.844001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:20.845026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.845512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.845996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] PATH_INFO: `/v3/users/cb88baeb87eb4ff7ad648850f690ed9c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.846123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:20.846758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:20.847055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:20.847394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:20.865242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:20.869778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] RBAC: Authorizing `identity:get_user(user_id=cb88baeb87eb4ff7ad648850f690ed9c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:20.870484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:20.871385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:20.872244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1052] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 9 06:51:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:20.872924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:20.879545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 519/1053] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:20 2022] GET /identity/v3/users/cb88baeb87eb4ff7ad648850f690ed9c => generated 285 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:20.879701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:20.880326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:20.910703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:20.975686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:20.989841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f4e9a-5274-49d4-96a7-37ce072d67b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.002009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643744-00e4-45ff-adf2-3be6f4a31499 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.028804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.029719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.030174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.030737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.031716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:21.032202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.033785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.035908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.036599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.036883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.037154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.044804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1054] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:51:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:21.050027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.115090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.116582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.137580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.153670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc65db-1fe3-46d5-80f9-2b2013e3b437 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.192004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.192632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.192931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.193179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.202045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.203646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.205446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:21.220241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:21.221420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:21.222316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:21.222316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:21.222316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:21.222316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1055] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:21.227003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.238855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 520/1056] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 06:51:21.240881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.328685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ae5a-e85d-496d-a751-8d50cab284a6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.340320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61059-c614-40ad-84f3-e53dab16076b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.361822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.362420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.362704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.362972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.363625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:21.363932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.365280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ffbf4-f06a-41cf-9d9c-8ee5e59f0707 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.372013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.374382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb146a-d789-4f15-9be5-70bc2eb26adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.375020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb146a-d789-4f15-9be5-70bc2eb26adb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.375309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb146a-d789-4f15-9be5-70bc2eb26adb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.375589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb146a-d789-4f15-9be5-70bc2eb26adb None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.376736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb146a-d789-4f15-9be5-70bc2eb26adb None admin] RBAC: Authorizing `identity:get_project(project_id=29a62f9cbdd242a09d5eede42df6c950)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.378056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1057] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:21.378412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb146a-d789-4f15-9be5-70bc2eb26adb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.379230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d430-a0bd-44b6-a97d-c0b9f47c81a1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.379513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d430-a0bd-44b6-a97d-c0b9f47c81a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.379813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d430-a0bd-44b6-a97d-c0b9f47c81a1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.381144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1058] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:21.383547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0d430-a0bd-44b6-a97d-c0b9f47c81a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.385548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 521/1059] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:21.386736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.387874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.387874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.405334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:21.406536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:21.445533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.472541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 522/1060] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:21.473359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.483468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0d430-a0bd-44b6-a97d-c0b9f47c81a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.518558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.519180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.519457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.519756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.526870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.528330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:21.539496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:21.545695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1061] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:21.546683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.576289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11648ca3-01cf-4070-b158-073c6824ec30 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.597898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.609934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.610537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.610814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.611084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.611832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=29a62f9cbdd242a09d5eede42df6c950, user.id=cb88baeb87eb4ff7ad648850f690ed9c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:21.612135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.613549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.619925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 523/1062] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/role_assignments?user.id=cb88baeb87eb4ff7ad648850f690ed9c&scope.project.id=29a62f9cbdd242a09d5eede42df6c950&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:21.620698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.645073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4b7a6-8c20-4f90-8c83-d14c2d7dd72c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.664921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.685195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.686108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.686540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.686957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.687782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.689183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:21.699240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:21.700355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:21.701437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1063] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:21.706646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.737519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85300c01-6330-48a5-91c7-d60a64bcecb6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.778232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.778842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.779123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.779398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.780161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:21.781110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.782771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.808545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 524/1064] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:21.814244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.821198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b5bff-e84a-42dc-a8ee-eaf616b28e2d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.844419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.859783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.860467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.860934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.861275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.861918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:21.862218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.863587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:21.869541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1065] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:21.874898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:21.928484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:21.937881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b217ed0-6dec-4439-b716-ef536a400b6b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.974433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:21.975061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:21.975343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:21.975632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:21.975746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253e70-991b-4551-9d52-be47d8cdd5ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:21.986890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:21.988481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:22.005060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:22.006028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:22.006907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:22.006907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:22.006907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:22.006907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 525/1066] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:22.011892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:22.013645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:22.014269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:22.014530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:22.014805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:22.015426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:22.015758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:22.017113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.022236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1067] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:22.027699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:22.081467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:22.127398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703e64-67fa-44c2-ab87-f2e641155b86 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:22.138757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb48f1-d586-4ade-ad43-f7f172050fc3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:22.163897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:22.164502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:22.164778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:22.165047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:22.166336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:22.166889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:22.168400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.174651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:22.175257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:22.175540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:22.175837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:22.176490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:22.177819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.181242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 526/1068] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:22.186447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:22.197437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:22.206073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1069] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:51:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:51:22.299824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed759238-82ac-42d6-b4d9-7e8375c16286 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:22.344664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:22.345317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:22.345602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:22.345877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:22.353880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:22.355477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:22.368365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:22.369323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:22.370334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 527/1070] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:22.375992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:22.480382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2eef1-4c09-40ce-8fe6-b8235f13b210 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:22.487413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:22.518430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:22.519014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:22.519272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:22.519561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:22.520248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:22.520591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:22.522010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a639e-601e-46e7-81ee-8c595841b471 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.539103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1071] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:22.540501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:22.608354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:22.649145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0dbd7-59e2-4e03-a0c3-1a0c502748b3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:22.684820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dca64-2ea9-45a1-8623-5e5d74b23c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:22.685549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dca64-2ea9-45a1-8623-5e5d74b23c1d None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:22.685880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dca64-2ea9-45a1-8623-5e5d74b23c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:22.686414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dca64-2ea9-45a1-8623-5e5d74b23c1d None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/d80a908c1da74b3fb3e82c8277556e81/roles/eb9f9897e742453580668a6cbb647017` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:22.688341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dca64-2ea9-45a1-8623-5e5d74b23c1d None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=d80a908c1da74b3fb3e82c8277556e81, role_id=eb9f9897e742453580668a6cbb647017)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:22.689772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dca64-2ea9-45a1-8623-5e5d74b23c1d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:22.707878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 528/1072] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:51:22 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/d80a908c1da74b3fb3e82c8277556e81/roles/eb9f9897e742453580668a6cbb647017 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:51:23.638069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69437b5-afec-4526-b356-e8afd5b7a95a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:23.638617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69437b5-afec-4526-b356-e8afd5b7a95a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:23.638617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69437b5-afec-4526-b356-e8afd5b7a95a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:23.640137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1073] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:23.645197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:23.645585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:23.645767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:23.663442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:23.664666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:23.731056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 529/1074] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:23.754138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69437b5-afec-4526-b356-e8afd5b7a95a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:23.863678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69437b5-afec-4526-b356-e8afd5b7a95a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:23.897655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:23.898206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:23.898485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:23.898756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:23.909337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:23.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:23.914874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:23.916724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:23.919932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1075] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:23.921744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:23.979211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.027390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08964c6a-4902-42c7-a695-5c0287a73f37 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:24.061264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:24.061869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.062148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.062419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.063076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:24.063377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:24.064770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:24.078497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 530/1076] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:24.080184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.110178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.189876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63603ac1-cdb9-47ce-838b-cc4ed3dcce31 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:24.224103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f941d-f84a-4ae0-989b-4fc1fc1b95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:24.224739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f941d-f84a-4ae0-989b-4fc1fc1b95ca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.225018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f941d-f84a-4ae0-989b-4fc1fc1b95ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.225290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f941d-f84a-4ae0-989b-4fc1fc1b95ca None admin] PATH_INFO: `/v3/users/cb88baeb87eb4ff7ad648850f690ed9c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.226407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f941d-f84a-4ae0-989b-4fc1fc1b95ca None admin] RBAC: Authorizing `identity:get_user(user_id=cb88baeb87eb4ff7ad648850f690ed9c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:24.227861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f941d-f84a-4ae0-989b-4fc1fc1b95ca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:24.229701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1077] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:24 2022] GET /identity/v3/users/cb88baeb87eb4ff7ad648850f690ed9c => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:24.235342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.258067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba350b-c745-47c6-b9a0-0a0eac865a1c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.258351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba350b-c745-47c6-b9a0-0a0eac865a1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.258627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba350b-c745-47c6-b9a0-0a0eac865a1c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.260750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1078] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:24.265425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73297435-b309-436b-83d2-274e866a2e92 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.265717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73297435-b309-436b-83d2-274e866a2e92 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.265989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73297435-b309-436b-83d2-274e866a2e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.284680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-73297435-b309-436b-83d2-274e866a2e92 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:24.285865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73297435-b309-436b-83d2-274e866a2e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:24.345807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ef525-64b8-4b0d-8d48-ec7c7bd1b9c8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:24.358711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1079] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:24.363177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.382352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:24.382533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73297435-b309-436b-83d2-274e866a2e92 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.382984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.383259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.383528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.385375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] RBAC: Authorizing `identity:get_project(project_id=29a62f9cbdd242a09d5eede42df6c950)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:24.386807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:24.393328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 531/1080] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:24 2022] GET /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:24.394119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.412387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.497004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73297435-b309-436b-83d2-274e866a2e92 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:24.501995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec739af-70ea-4075-94f4-a52edf7ef32e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:24.535931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:24.536369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf21894-3cfb-483d-8598-48d447fad2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:24.537025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf21894-3cfb-483d-8598-48d447fad2e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.537301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf21894-3cfb-483d-8598-48d447fad2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.537573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf21894-3cfb-483d-8598-48d447fad2e1 None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950/users/cb88baeb87eb4ff7ad648850f690ed9c/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.537691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.538156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.538694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.539707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf21894-3cfb-483d-8598-48d447fad2e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a62f9cbdd242a09d5eede42df6c950, user_id=cb88baeb87eb4ff7ad648850f690ed9c, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:24.541238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf21894-3cfb-483d-8598-48d447fad2e1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:24.567695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 532/1081] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:51:24 2022] PUT /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950/users/cb88baeb87eb4ff7ad648850f690ed9c/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:51:24.619495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:24.621406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:24.649114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.737809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1082] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 06:51:24.739679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c964-951b-4576-a34d-32753527c9d9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.740121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.740203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c964-951b-4576-a34d-32753527c9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.740281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c964-951b-4576-a34d-32753527c9d9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.741583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 533/1083] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:24.747385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.747692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.747923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.766551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:24.767780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:24.811667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.854208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85e09c-df5b-4b8c-9890-e20497c78d0f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:24.855958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 534/1084] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:24.873234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.882127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:24.897242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:24.897881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:24.898159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:24.898430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:24.910457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:24.911930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:24.928863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:24.929883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:24.930777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:24.930777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:24.930777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:24.930777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1085] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:24.936838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:24.981821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7545f9-794a-4b1a-a2ed-edd950bf98de None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.020746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.021295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.021998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.022277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.036848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:25.042508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:25.043443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:25.044352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 535/1086] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 9 06:51:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:25.050756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54bfcb-8498-492f-887a-6eb9be80f178 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.051984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:25.090600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:25.090792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.091214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.092136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.092136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.092426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:25.092737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:25.094490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:25.109097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1087] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:25.173510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211ec60-c1cf-429b-a394-b663fb192e27 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.211091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.211749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.212032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.212305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.213784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:25.214089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:25.215497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:25.230325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 536/1088] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:51:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:25.232036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:25.270159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:25.338432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f726b07-8918-42e9-ae21-32a2202151c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.372347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.373028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.373305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.373628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.384167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:25.385792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:25.401139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:25.402329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:25.403372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:25.403372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:25.403372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:25.404045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1089] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:25.409598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:25.501606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:25.508528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0252d57-4db5-4e04-accf-2e86c4e65ccb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.542102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.542703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.542988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.543268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.543939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:25.544244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:25.545596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:25.557318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 537/1090] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:25.562848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:25.649336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:25.661682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470cac4-a0df-42a8-b10c-cbc14b36f383 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.697050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.697829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.698110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.698389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.705912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:25.707333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:51:25.718939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:25.719999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:25.721548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1091] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 9 06:51:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:25.725856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:25.824370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d1699-da72-49ea-9cb3-09ee6e773a03 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:25.858982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:25.859584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:25.859890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:25.860165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:25.860883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:25.861188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:25.862605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:25.880152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 538/1092] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 9 06:51:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:25.885990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:25.983733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0c38-4304-48f8-860c-f62523e321fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.016048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.016652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.016931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.017207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.035156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.083718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:26.085155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:26.183474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1093] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:26.189216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.204606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.290172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae77cfb-967a-4ba1-9a26-6693f13313c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.323911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.324540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.324820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.325094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.334900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:26.336375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:26.351141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:26.352266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:51:26.353541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:26.353541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:51:26.353541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:26.355747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 539/1094] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:26.357974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.406377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.456448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c5fac-791f-4923-bba2-a78e6e0bb4bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.489877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.490483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.490765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.491040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.491714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:26.492021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:26.493376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330a2f-1b16-4d08-a792-a37e5ad8d8c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:26.504956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1095] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:26.510842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.576791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.593700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8001292-3ba9-4fc1-80a2-fcf7c61e5181 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.593985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8001292-3ba9-4fc1-80a2-fcf7c61e5181 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.594269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8001292-3ba9-4fc1-80a2-fcf7c61e5181 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.595624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1096] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:51:26.600277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.600564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.600839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.614637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bae1f3-588f-4e9f-aa59-033da0eda2d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.618812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:26.620204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:26.649443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.650592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.650886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.651157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.657780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:26.659219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:51:26.671230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:26.672183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:26.673050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 540/1097] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:26.678041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.694436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1098] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:26.717892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.757308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.777201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e1f4-8d5d-4d93-a35b-9dd4ba0e756e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.802523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.811942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.812538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.812818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.813091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.813778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:26.814083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:26.815512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:26.817569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab15d04-0f82-4cf1-8ca6-b00a72a53442 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.827536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 541/1099] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:26.833378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.852550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.853766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.854047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.854326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.864667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:26.869445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:26.870579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:26.871404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1100] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:51:26.876521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:26.936748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fefd6-5586-49c4-baab-9c3593384b51 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.970402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:26.970993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:26.971271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:26.971549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:26.971791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:26.972407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] RBAC: Adding query filter params (role.id=eb9f9897e742453580668a6cbb647017, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=d80a908c1da74b3fb3e82c8277556e81) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:26.972586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:26.974003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:26.976062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb17e06-f2c0-4c23-86ee-55360b6e6aba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:26.980616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 542/1101] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/role_assignments?user.id=d80a908c1da74b3fb3e82c8277556e81&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=eb9f9897e742453580668a6cbb647017 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:27.017427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:27.018308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:27.018715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:27.019111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:27.020147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:27.020568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:27.022580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:27.034251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1102] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:27.040787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:27.094278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:27.148254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cd001-e952-48f5-882d-b5695464310b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:27.185576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:27.186209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:27.186489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:27.186767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] PATH_INFO: `/v3/users/cb88baeb87eb4ff7ad648850f690ed9c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:27.187925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=cb88baeb87eb4ff7ad648850f690ed9c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:27.189342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:27.190911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 543/1103] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:27 2022] GET /identity/v3/users/cb88baeb87eb4ff7ad648850f690ed9c => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:27.197541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:27.301782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a958d-746b-4118-a590-14b0566f4270 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:27.336661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:27.337264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:27.338207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:27.338700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:27.366815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:27.408017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:27.409460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:27.512153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1104] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:27.517908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:27.620849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf3a5-0a5a-41e4-8c76-af7f6ab18ab3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:27.638608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:27.655884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57739a-e9a2-426f-a007-868d38754f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:27.656519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57739a-e9a2-426f-a007-868d38754f94 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:27.656798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57739a-e9a2-426f-a007-868d38754f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:27.657073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57739a-e9a2-426f-a007-868d38754f94 None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:27.658211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57739a-e9a2-426f-a007-868d38754f94 None admin] RBAC: Authorizing `identity:get_project(project_id=29a62f9cbdd242a09d5eede42df6c950)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:27.659666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57739a-e9a2-426f-a007-868d38754f94 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:27.661296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 544/1105] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:27 2022] GET /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:51:27.667460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:27.728627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:27.769249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ead21-502a-4b2c-87e9-8d037d8d1a96 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:27.803348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:27.803997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:27.804275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:27.804551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:27.805282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=29a62f9cbdd242a09d5eede42df6c950, user.id=cb88baeb87eb4ff7ad648850f690ed9c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:27.805586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:27.807025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d50032-405c-4694-a581-b6e29b539079 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:27.812968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:27 2022] GET /identity/v3/role_assignments?user.id=cb88baeb87eb4ff7ad648850f690ed9c&scope.project.id=29a62f9cbdd242a09d5eede42df6c950&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:29.816376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8d00-c5cb-44fb-8f31-d114bcc7eb42 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:29.816900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8d00-c5cb-44fb-8f31-d114bcc7eb42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:29.816994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8d00-c5cb-44fb-8f31-d114bcc7eb42 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:29.818261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 545/1107] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:29.824225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:29.824623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:29.824774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:29.842506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:29.843713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:29.908233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:29.933189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8d00-c5cb-44fb-8f31-d114bcc7eb42 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:30.032319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8d00-c5cb-44fb-8f31-d114bcc7eb42 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:30.065020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348cfe5-5e6b-477f-8e8c-3bd4248e3df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:30.065629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348cfe5-5e6b-477f-8e8c-3bd4248e3df6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:30.065910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348cfe5-5e6b-477f-8e8c-3bd4248e3df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:30.066183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348cfe5-5e6b-477f-8e8c-3bd4248e3df6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:30.067330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348cfe5-5e6b-477f-8e8c-3bd4248e3df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:30.068736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348cfe5-5e6b-477f-8e8c-3bd4248e3df6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:30.070263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 546/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:30.075967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:30.128970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:30.171759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d83ee-b63c-412a-acc4-1ed14dd19979 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:30.204599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf4d8-886e-4ae7-a4e8-c6fcaf63a4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:30.205190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf4d8-886e-4ae7-a4e8-c6fcaf63a4da None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:30.205469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf4d8-886e-4ae7-a4e8-c6fcaf63a4da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:30.205739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf4d8-886e-4ae7-a4e8-c6fcaf63a4da None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:30.206376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf4d8-886e-4ae7-a4e8-c6fcaf63a4da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:30.207816 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf4d8-886e-4ae7-a4e8-c6fcaf63a4da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:30.229181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:51:30 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:32.221814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac2cf5-bd8a-4eb5-8387-211451077d8c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:32.222343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac2cf5-bd8a-4eb5-8387-211451077d8c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:32.222343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac2cf5-bd8a-4eb5-8387-211451077d8c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:32.223717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 547/1111] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:32.229320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:32.229751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:32.229844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:32.248206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:32.249384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:32.313489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:32.336608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac2cf5-bd8a-4eb5-8387-211451077d8c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:32.391976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:32.436743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac2cf5-bd8a-4eb5-8387-211451077d8c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:32.468635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:32.469286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:32.469566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:32.469839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] PATH_INFO: `/v3/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:32.471261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] RBAC: Authorizing `identity:get_role(role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:32.472702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:32.473963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 548/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:32 2022] GET /identity/v3/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:32.480756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:32.522189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:32.579290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183293f-0d5d-4653-8666-820205ad2273 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:32.612665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:32.613283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:32.613566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:32.613897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] PATH_INFO: `/v3/users/0c17e4b1ccb54ae1bbd5704bfd78a76e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:32.626838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] RBAC: Authorizing `identity:get_user(user_id=0c17e4b1ccb54ae1bbd5704bfd78a76e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:32.628542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:32.630121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:32 2022] GET /identity/v3/users/0c17e4b1ccb54ae1bbd5704bfd78a76e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:32.636028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:32.734651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac61fce-fd41-4887-b73d-c4a436dbea23 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:32.756220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:32.768120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:32.768720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:32.769003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:32.769281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:32.834402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:32.835875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:32.932220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 549/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:32.938332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:33.030787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:33.034588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7f4ad-ca2d-4141-996d-0f9ac17d0933 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:33.067623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46cebd-d227-4a93-a941-1d98fe89b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:33.068257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46cebd-d227-4a93-a941-1d98fe89b62a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:33.068539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46cebd-d227-4a93-a941-1d98fe89b62a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:33.068815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46cebd-d227-4a93-a941-1d98fe89b62a None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:33.069920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46cebd-d227-4a93-a941-1d98fe89b62a None admin] RBAC: Authorizing `identity:get_project(project_id=29a62f9cbdd242a09d5eede42df6c950)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:33.071344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46cebd-d227-4a93-a941-1d98fe89b62a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:33.073647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:32 2022] GET /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:33.080033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:33.104436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:33.179470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999206-2ad3-4757-b5f5-3c1c8183721c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:33.212547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:33.213142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:33.213428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:33.213700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:33.214422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] RBAC: Adding query filter params (role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=29a62f9cbdd242a09d5eede42df6c950, user.id=0c17e4b1ccb54ae1bbd5704bfd78a76e) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:33.214729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:33.216180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef06a-66bb-46a0-a160-1ca3c6e74db8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:33.221352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 550/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:33 2022] GET /identity/v3/role_assignments?user.id=0c17e4b1ccb54ae1bbd5704bfd78a76e&scope.project.id=29a62f9cbdd242a09d5eede42df6c950&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:35.175053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908a1e4-9976-4523-911b-937b248f51f6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:35.175564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908a1e4-9976-4523-911b-937b248f51f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:35.175721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908a1e4-9976-4523-911b-937b248f51f6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:35.176954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1118] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:35.182553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:35.182951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:35.183097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:35.201333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:35.202508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:35.266614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 551/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:35.288427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908a1e4-9976-4523-911b-937b248f51f6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:35.385214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908a1e4-9976-4523-911b-937b248f51f6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:35.417257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:35.417867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:35.418141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:35.418410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] PATH_INFO: `/v3/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:35.419789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] RBAC: Authorizing `identity:get_role(role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:35.421123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:35.422410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:35 2022] GET /identity/v3/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:35.428403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:35.445963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:35.525603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eda44-072d-4b9d-aa61-30fd7d20aa2f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:35.559170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:35.559820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:35.560102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:35.560372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] PATH_INFO: `/v3/users/0c17e4b1ccb54ae1bbd5704bfd78a76e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:35.561548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] RBAC: Authorizing `identity:get_user(user_id=0c17e4b1ccb54ae1bbd5704bfd78a76e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:35.562972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:35.564572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 552/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:35 2022] GET /identity/v3/users/0c17e4b1ccb54ae1bbd5704bfd78a76e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:35.570377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:35.604678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:35.668905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba809275-832e-4ab8-88a9-7deee45fe068 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:35.701311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23272c-4139-4d60-8fdc-64c25a442edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:35.701872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23272c-4139-4d60-8fdc-64c25a442edb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:35.702153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23272c-4139-4d60-8fdc-64c25a442edb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:35.702428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23272c-4139-4d60-8fdc-64c25a442edb None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:35.703529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23272c-4139-4d60-8fdc-64c25a442edb None admin] RBAC: Authorizing `identity:get_project(project_id=29a62f9cbdd242a09d5eede42df6c950)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:35.704963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23272c-4139-4d60-8fdc-64c25a442edb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:35.706567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:35 2022] GET /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:35.713048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:35.794285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:35.809807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc8d2f-d7be-4fc1-b5df-81c06aa15b6a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:35.842721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60709e49-a642-4ec5-bf67-03c9a11157a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:35.843378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60709e49-a642-4ec5-bf67-03c9a11157a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:35.843678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60709e49-a642-4ec5-bf67-03c9a11157a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:35.843952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60709e49-a642-4ec5-bf67-03c9a11157a5 None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950/users/0c17e4b1ccb54ae1bbd5704bfd78a76e/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:35.846031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60709e49-a642-4ec5-bf67-03c9a11157a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a62f9cbdd242a09d5eede42df6c950, user_id=0c17e4b1ccb54ae1bbd5704bfd78a76e, role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:35.847438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60709e49-a642-4ec5-bf67-03c9a11157a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:35.860481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 553/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:51:35 2022] PUT /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950/users/0c17e4b1ccb54ae1bbd5704bfd78a76e/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:51:37.810311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee86a5-69ea-4da9-8489-beaacce5ab97 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:37.810870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee86a5-69ea-4da9-8489-beaacce5ab97 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:37.810870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee86a5-69ea-4da9-8489-beaacce5ab97 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:37.812227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1124] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:37.817402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:37.817817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:37.817947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:37.836280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:37.838332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:37.904426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 554/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 06:51:37.927275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee86a5-69ea-4da9-8489-beaacce5ab97 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:38.026521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee86a5-69ea-4da9-8489-beaacce5ab97 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:38.028045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:38.060775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:38.061330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:38.061612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:38.061882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] PATH_INFO: `/v3/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:38.063242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] RBAC: Authorizing `identity:get_role(role_id=bf922a48d8bb4dae91a0dd3d2d94b6fa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:38.064660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:38.065959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:37 2022] GET /identity/v3/roles/bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:38.072456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:38.143525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:38.168600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2353bf-9467-4cb3-8fed-eca1f15836ff None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:38.201538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:38.202152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:38.202428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:38.202697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] PATH_INFO: `/v3/users/0c17e4b1ccb54ae1bbd5704bfd78a76e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:38.203824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] RBAC: Authorizing `identity:get_user(user_id=0c17e4b1ccb54ae1bbd5704bfd78a76e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:38.205242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:38.206784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 555/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:38 2022] GET /identity/v3/users/0c17e4b1ccb54ae1bbd5704bfd78a76e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:38.212635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:38.313995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c60655-5806-421b-a162-d01343e5b10b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:38.346714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:38.347308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:38.347585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:38.347882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:38.386412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:38.414057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:38.415540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:38.510631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:38.516546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:38.611886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fa32e-3e37-4282-9f6e-027fd944f5e8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:38.645606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14274214-8edd-422e-9405-a46e920626a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:38.646227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274214-8edd-422e-9405-a46e920626a3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:38.646510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274214-8edd-422e-9405-a46e920626a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:38.646813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274214-8edd-422e-9405-a46e920626a3 None admin] PATH_INFO: `/v3/projects/29a62f9cbdd242a09d5eede42df6c950` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:38.647962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14274214-8edd-422e-9405-a46e920626a3 None admin] RBAC: Authorizing `identity:get_project(project_id=29a62f9cbdd242a09d5eede42df6c950)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:38.649533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14274214-8edd-422e-9405-a46e920626a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:38.651051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 556/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:38 2022] GET /identity/v3/projects/29a62f9cbdd242a09d5eede42df6c950 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:38.657013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:38.723704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:38.755501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3087-f325-4781-b9fe-f629be9d1cd3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:38.788551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:38.789147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:38.789433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:38.789705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:38.790553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] RBAC: Adding query filter params (role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa, scope.project.id=29a62f9cbdd242a09d5eede42df6c950, user.id=0c17e4b1ccb54ae1bbd5704bfd78a76e) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:38.790855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:38.792295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb09815-0436-4198-b9e1-24ba7fb1ffd5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:38.797849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:38 2022] GET /identity/v3/role_assignments?user.id=0c17e4b1ccb54ae1bbd5704bfd78a76e&scope.project.id=29a62f9cbdd242a09d5eede42df6c950&role.id=bf922a48d8bb4dae91a0dd3d2d94b6fa => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:40.835536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ace821-30bb-4f9a-99fc-80e8a1eccb9a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:40.836118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ace821-30bb-4f9a-99fc-80e8a1eccb9a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:40.836118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ace821-30bb-4f9a-99fc-80e8a1eccb9a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:40.838023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 557/1131] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:40.843290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:40.843737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:40.843871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:40.862056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:40.863242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:40.927793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:40.950402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ace821-30bb-4f9a-99fc-80e8a1eccb9a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:40.963250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:41.050977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ace821-30bb-4f9a-99fc-80e8a1eccb9a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:41.083333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c9e81-ec9c-4d4e-87f0-f64bb48f278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:41.083979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c9e81-ec9c-4d4e-87f0-f64bb48f278c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:41.084257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c9e81-ec9c-4d4e-87f0-f64bb48f278c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:41.084526 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c9e81-ec9c-4d4e-87f0-f64bb48f278c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:41.085671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c9e81-ec9c-4d4e-87f0-f64bb48f278c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:41.087561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c9e81-ec9c-4d4e-87f0-f64bb48f278c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:41.089129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 558/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:41.094750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:41.153882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:41.189953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb7dbe-e9c2-40e3-a183-91aaa5841f84 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:41.222577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057449cd-22d1-42b7-a02c-7f77f8cd447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:41.223181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057449cd-22d1-42b7-a02c-7f77f8cd447f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:41.223464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057449cd-22d1-42b7-a02c-7f77f8cd447f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:41.223774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057449cd-22d1-42b7-a02c-7f77f8cd447f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:41.224426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057449cd-22d1-42b7-a02c-7f77f8cd447f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:41.225839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057449cd-22d1-42b7-a02c-7f77f8cd447f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:41.252650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:51:41 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:43.270834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70aa12-d336-4f87-bb86-8a328549179f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:43.271384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70aa12-d336-4f87-bb86-8a328549179f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:43.271384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70aa12-d336-4f87-bb86-8a328549179f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:43.272755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 559/1135] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:43.278529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:43.278923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:43.279086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:43.297424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:43.298623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:43.362575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:43.386949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70aa12-d336-4f87-bb86-8a328549179f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:43.463394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:43.485907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70aa12-d336-4f87-bb86-8a328549179f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:43.520111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a62a44-ac68-4084-b1fd-ec434e1ed4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:43.520983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a62a44-ac68-4084-b1fd-ec434e1ed4fd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:43.521383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a62a44-ac68-4084-b1fd-ec434e1ed4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:43.521785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a62a44-ac68-4084-b1fd-ec434e1ed4fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:43.523224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a62a44-ac68-4084-b1fd-ec434e1ed4fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:43.525929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a62a44-ac68-4084-b1fd-ec434e1ed4fd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:43.527970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 560/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:51:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:43.533707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:43.578569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:43.629850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35607d8-ad71-47e8-905b-c6713dcd6ec4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:43.662937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4ab78-85e1-499a-9bf4-e434080bd81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:43.663543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ab78-85e1-499a-9bf4-e434080bd81c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:43.663851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ab78-85e1-499a-9bf4-e434080bd81c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:43.664124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ab78-85e1-499a-9bf4-e434080bd81c None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:43.664793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4ab78-85e1-499a-9bf4-e434080bd81c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:43.666194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4ab78-85e1-499a-9bf4-e434080bd81c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:43.686490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:51:43 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:45.698616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab37d-d5a9-4825-a773-22c3e16f7418 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:45.699250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab37d-d5a9-4825-a773-22c3e16f7418 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:45.699250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab37d-d5a9-4825-a773-22c3e16f7418 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:45.700522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 561/1139] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:45.705509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:45.705953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:45.706056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:45.724167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:45.725342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:45.792005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:45.814450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab37d-d5a9-4825-a773-22c3e16f7418 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:45.914407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab37d-d5a9-4825-a773-22c3e16f7418 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:45.948041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:45.948711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:45.948963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:45.949268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:45.959923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:45.966761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:45.967970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:45.969205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 562/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:51:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:45.975621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:46.006053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:46.074822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d6a69-5032-4871-83e8-9748f211ad05 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:46.107866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:46.108465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:46.108748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:46.109031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:46.109694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:46.109999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:46.111360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:46.119681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:46.125474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:46.135517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:46.223294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac1a80-2d39-43fb-b332-b7cda4d2355e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:46.256246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:46.256859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:46.257139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:46.257412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] PATH_INFO: `/v3/users/fc7b9ed72547469b8944b22a0ffa21cc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:46.269953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] RBAC: Authorizing `identity:get_user(user_id=fc7b9ed72547469b8944b22a0ffa21cc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:46.271382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:46.273001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 563/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:46 2022] GET /identity/v3/users/fc7b9ed72547469b8944b22a0ffa21cc => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:46.279418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:46.371309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:46.378390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dc1be-5770-41a1-99cd-a44c19ee491f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:46.411059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:46.411692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:46.411979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:46.412258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:46.477832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:46.479208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:46.567535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:46.575805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:46.581640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:46.643236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:46.680226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62409407-6ad6-4962-af1c-8b5853618849 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:46.713464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dbf45-7ed3-4d5a-a7da-63d47c0ec81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:46.714079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dbf45-7ed3-4d5a-a7da-63d47c0ec81d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:46.714359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dbf45-7ed3-4d5a-a7da-63d47c0ec81d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:46.714632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dbf45-7ed3-4d5a-a7da-63d47c0ec81d None admin] PATH_INFO: `/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:46.715805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034dbf45-7ed3-4d5a-a7da-63d47c0ec81d None admin] RBAC: Authorizing `identity:get_project(project_id=35ab163ca6784f089e4a4caf5cd58b6a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:46.717199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034dbf45-7ed3-4d5a-a7da-63d47c0ec81d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:46.718781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 564/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:46 2022] GET /identity/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:46.724739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:46.821428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126fcec-bcf7-4900-85a0-62fe5aa4c276 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:46.855105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:46.855734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:46.856017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:46.856289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:46.857011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=35ab163ca6784f089e4a4caf5cd58b6a, user.id=fc7b9ed72547469b8944b22a0ffa21cc) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:46.857323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:46.858806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2db8f9-684a-4043-8052-fdf8575eb871 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:46.864066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:46 2022] GET /identity/v3/role_assignments?user.id=fc7b9ed72547469b8944b22a0ffa21cc&scope.project.id=35ab163ca6784f089e4a4caf5cd58b6a&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:48.821493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8de1-93bd-465a-ba48-109bac2ad50b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:48.822052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8de1-93bd-465a-ba48-109bac2ad50b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:48.822052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d8de1-93bd-465a-ba48-109bac2ad50b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:48.823391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 565/1147] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:48.828762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:48.829192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:48.829291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:48.847705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:48.848890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:48.900710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:48.914137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:48.935833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d8de1-93bd-465a-ba48-109bac2ad50b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:49.035020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d8de1-93bd-465a-ba48-109bac2ad50b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:49.064328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:49.069422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:49.070034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:49.070311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:49.070581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:49.080433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:49.085016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:49.086100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:49.088035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 566/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:49.092537 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:49.186686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a3369-e505-4b4c-b820-a401cd1a9ac2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:49.218950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:49.219544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:49.219849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:49.220118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:49.220772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:49.221073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:49.222419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:49.230876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:49.236572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:49.332258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6153ad-9504-4746-a2de-0a6bf2e7b112 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:49.365316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:49.365922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:49.366198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:49.366464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] PATH_INFO: `/v3/users/fc7b9ed72547469b8944b22a0ffa21cc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:49.367657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] RBAC: Authorizing `identity:get_user(user_id=fc7b9ed72547469b8944b22a0ffa21cc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:49.369082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:49.370610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 567/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:49 2022] GET /identity/v3/users/fc7b9ed72547469b8944b22a0ffa21cc => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:49.376311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:49.425548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:49.474028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89908c00-ca68-4c4a-9468-10034fc9639f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:49.506522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e44664-ef82-4599-893d-79318e5652a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:49.507142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e44664-ef82-4599-893d-79318e5652a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:49.507424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e44664-ef82-4599-893d-79318e5652a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:49.507731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e44664-ef82-4599-893d-79318e5652a1 None admin] PATH_INFO: `/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:49.509568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e44664-ef82-4599-893d-79318e5652a1 None admin] RBAC: Authorizing `identity:get_project(project_id=35ab163ca6784f089e4a4caf5cd58b6a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:49.510989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e44664-ef82-4599-893d-79318e5652a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:49.512632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:49 2022] GET /identity/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:49.518541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:49.575958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:49.615235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0bac-8f47-4a63-8cb4-cd7c2778eae3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:49.647857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221c376-3720-45c7-bf44-d9dd8e8532ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:49.648495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c376-3720-45c7-bf44-d9dd8e8532ac None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:49.649415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c376-3720-45c7-bf44-d9dd8e8532ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:49.649707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c376-3720-45c7-bf44-d9dd8e8532ac None admin] PATH_INFO: `/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a/users/fc7b9ed72547469b8944b22a0ffa21cc/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:49.651566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221c376-3720-45c7-bf44-d9dd8e8532ac None admin] RBAC: Authorizing `identity:create_grant(project_id=35ab163ca6784f089e4a4caf5cd58b6a, user_id=fc7b9ed72547469b8944b22a0ffa21cc, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:49.652999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221c376-3720-45c7-bf44-d9dd8e8532ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:49.666154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 568/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:51:49 2022] PUT /identity/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a/users/fc7b9ed72547469b8944b22a0ffa21cc/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:51:51.622603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9f6b-0d9d-42e2-94d6-6d328b3095f7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:51.623130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9f6b-0d9d-42e2-94d6-6d328b3095f7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:51.623130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9f6b-0d9d-42e2-94d6-6d328b3095f7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:51.624536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1154] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:51.629953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83105-3991-42dc-965c-b37b5330670d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:51.630353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83105-3991-42dc-965c-b37b5330670d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:51.630499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83105-3991-42dc-965c-b37b5330670d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:51.648121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-99c83105-3991-42dc-965c-b37b5330670d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:51.649894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-99c83105-3991-42dc-965c-b37b5330670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:51.714092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 569/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:51.737758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e9f6b-0d9d-42e2-94d6-6d328b3095f7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:51.838104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e9f6b-0d9d-42e2-94d6-6d328b3095f7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:51.855731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:51.873279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:51.873848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:51.874131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:51.874408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:51.884748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:51:51.889423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:51.890492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:51:51.891364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:51.896616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83105-3991-42dc-965c-b37b5330670d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:51.944335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:51.993763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83105-3991-42dc-965c-b37b5330670d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:52.026804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:52.027400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:52.027716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:52.027996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:52.028659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:52.028963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:52.030318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:52.038081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 570/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:52.044389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:52.140686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be156-3182-4466-8422-ff8dbe071c8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:52.174640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:52.175254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:52.175536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:52.175837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] PATH_INFO: `/v3/users/fc7b9ed72547469b8944b22a0ffa21cc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:52.176951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] RBAC: Authorizing `identity:get_user(user_id=fc7b9ed72547469b8944b22a0ffa21cc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:52.178365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:52.180495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:51:52 2022] GET /identity/v3/users/fc7b9ed72547469b8944b22a0ffa21cc => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:52.186408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:52.283857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a39e3-5ff4-463e-bb45-679136544b8d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:52.296174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:52.318252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:52.318850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:52.319132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:52.319405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:52.385339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:52.386779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:52.480806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 571/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:52.486643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:52.532473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:52.585135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011f7c2-2665-479c-9d0a-fbf7a6454874 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:52.619216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc2638-0073-4c25-a0fb-260c4a638aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:52.619860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2638-0073-4c25-a0fb-260c4a638aec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:52.620143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2638-0073-4c25-a0fb-260c4a638aec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:52.620418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2638-0073-4c25-a0fb-260c4a638aec None admin] PATH_INFO: `/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:52.621537 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc2638-0073-4c25-a0fb-260c4a638aec None admin] RBAC: Authorizing `identity:get_project(project_id=35ab163ca6784f089e4a4caf5cd58b6a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:52.622959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc2638-0073-4c25-a0fb-260c4a638aec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:52.624612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:51:52 2022] GET /identity/v3/projects/35ab163ca6784f089e4a4caf5cd58b6a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:52.630923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:52.645255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:52.729370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179787bf-973f-443b-bf84-6fa8cba0fcf9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:52.762925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:52.763455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:52.763778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:52.764054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:52.764770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=35ab163ca6784f089e4a4caf5cd58b6a, user.id=fc7b9ed72547469b8944b22a0ffa21cc) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:52.765078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:52.766492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca9db6-9e18-48bc-86a4-afcdf429d5ef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:52.772141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 572/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:51:52 2022] GET /identity/v3/role_assignments?user.id=fc7b9ed72547469b8944b22a0ffa21cc&scope.project.id=35ab163ca6784f089e4a4caf5cd58b6a&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:54.766448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612aa21a-1020-4677-abef-cefa0abd9e85 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:54.766996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612aa21a-1020-4677-abef-cefa0abd9e85 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:54.766996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612aa21a-1020-4677-abef-cefa0abd9e85 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:54.768379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1162] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:54.773473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:54.774403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:54.774593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:54.792288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:54.793456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:54.859121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 573/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:54.881154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612aa21a-1020-4677-abef-cefa0abd9e85 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:54.978914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612aa21a-1020-4677-abef-cefa0abd9e85 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:54.992249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:55.012153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:55.012749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:55.013026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:55.013296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:55.078131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:55.079503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:55.174067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:51:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:51:55.180621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:55.188536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:55.276759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c96a3f-6b3d-44b9-bede-54f957cba097 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:55.309033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:55.309640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:55.309920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:55.310186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:55.316573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:55.317987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a765f965-0502-4bd1-af6d-6196f3936977 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:51:55.330793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:51:55.332038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:51:55.332909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 574/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:51:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:51:55.337912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:55.352784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:55.437282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c9546-a3f8-48ee-b495-60e5df326594 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:55.470271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:55.470867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:55.471148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:55.471422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:55.472090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:51:55.472397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:55.473758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb8e6e-590c-4979-8938-16c21ad824a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:55.481390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 9 06:51:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:57.427900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9687-fa91-4f9e-a88b-d348995025f4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:57.428488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9687-fa91-4f9e-a88b-d348995025f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:57.428488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9687-fa91-4f9e-a88b-d348995025f4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:57.429810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 575/1167] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:57.434868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e63b6-6071-4634-8dd5-139f678f9026 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:57.435280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e63b6-6071-4634-8dd5-139f678f9026 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:57.435496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e63b6-6071-4634-8dd5-139f678f9026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:57.453094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b7e63b6-6071-4634-8dd5-139f678f9026 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:57.454274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b7e63b6-6071-4634-8dd5-139f678f9026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:57.518786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:57.542165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9687-fa91-4f9e-a88b-d348995025f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:57.605211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:57.640362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9687-fa91-4f9e-a88b-d348995025f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:57.672004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6031e-0a70-46d7-a638-5c74914b21eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:57.672587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6031e-0a70-46d7-a638-5c74914b21eb None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:57.672862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6031e-0a70-46d7-a638-5c74914b21eb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:57.673131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6031e-0a70-46d7-a638-5c74914b21eb None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:57.673786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6031e-0a70-46d7-a638-5c74914b21eb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:57.675126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6031e-0a70-46d7-a638-5c74914b21eb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:57.690620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 576/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Tue Aug 9 06:51:57 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:51:59.714812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4699-718f-4102-858f-beed60962561 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:59.715367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4699-718f-4102-858f-beed60962561 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:59.715367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4699-718f-4102-858f-beed60962561 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:59.716741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1170] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:51:59.721735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:59.722164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:59.722269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:59.740755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:51:59.741927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:51:59.806675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 577/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:51:59.829746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf4699-718f-4102-858f-beed60962561 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:51:59.841151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:51:59.930940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf4699-718f-4102-858f-beed60962561 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:51:59.963011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f213231-137b-4a1a-8845-dfd9f28f8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:51:59.963649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f213231-137b-4a1a-8845-dfd9f28f8b68 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:51:59.963929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f213231-137b-4a1a-8845-dfd9f28f8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:51:59.964201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f213231-137b-4a1a-8845-dfd9f28f8b68 None admin] PATH_INFO: `/v3/domains/c79661815f8c4f158c35e66f96117651` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:51:59.974063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f213231-137b-4a1a-8845-dfd9f28f8b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c79661815f8c4f158c35e66f96117651)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:51:59.975435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f213231-137b-4a1a-8845-dfd9f28f8b68 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:51:59.977691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Aug 9 06:51:59 2022] GET /identity/v3/domains/c79661815f8c4f158c35e66f96117651 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:51:59.983532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:00.037930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:00.080896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae449a6-a3eb-4373-a9bb-b006a38fd1e1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:00.113916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c7e46-d03c-4d2a-8825-8c63d842428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:00.114512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7e46-d03c-4d2a-8825-8c63d842428e None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:00.114792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7e46-d03c-4d2a-8825-8c63d842428e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:00.115066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7e46-d03c-4d2a-8825-8c63d842428e None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:00.115735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c7e46-d03c-4d2a-8825-8c63d842428e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:00.117326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c7e46-d03c-4d2a-8825-8c63d842428e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:00.143266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 578/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:51:59 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:52:02.143742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b941c71-f118-4616-bfc5-474f73bd46e6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:02.144267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b941c71-f118-4616-bfc5-474f73bd46e6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:02.144267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b941c71-f118-4616-bfc5-474f73bd46e6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:02.145613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1174] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:52:02.150758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:02.151137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:02.151314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:02.169001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:52:02.170179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:52:02.235232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 579/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:52:02.260539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b941c71-f118-4616-bfc5-474f73bd46e6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:02.325676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:02.359044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b941c71-f118-4616-bfc5-474f73bd46e6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:02.392348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b01731-68cc-419c-a9df-c9ee33e441c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:02.392967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b01731-68cc-419c-a9df-c9ee33e441c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:02.393245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b01731-68cc-419c-a9df-c9ee33e441c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:02.393513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b01731-68cc-419c-a9df-c9ee33e441c4 None admin] PATH_INFO: `/v3/domains/c79661815f8c4f158c35e66f96117651` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:02.394682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b01731-68cc-419c-a9df-c9ee33e441c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c79661815f8c4f158c35e66f96117651)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:02.396086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b01731-68cc-419c-a9df-c9ee33e441c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:02.397653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Aug 9 06:52:02 2022] GET /identity/v3/domains/c79661815f8c4f158c35e66f96117651 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:02.403657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:02.442062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:02.501250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba18dc5-56bb-4511-8264-f2ffeea2d579 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:02.534161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3c287-5199-419d-9d10-70c5670adf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:02.534752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c287-5199-419d-9d10-70c5670adf3b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:02.535028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c287-5199-419d-9d10-70c5670adf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:02.535298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c287-5199-419d-9d10-70c5670adf3b None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:02.535959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3c287-5199-419d-9d10-70c5670adf3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:02.537371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3c287-5199-419d-9d10-70c5670adf3b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:02.559456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 580/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:52:02 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:52:04.590187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12d250-2a6e-4299-baef-05f9a0404b1c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:04.590715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12d250-2a6e-4299-baef-05f9a0404b1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:04.590715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12d250-2a6e-4299-baef-05f9a0404b1c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:04.592109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1178] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:52:04.597245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:04.597618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:04.597802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:04.616315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:52:04.618252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:52:04.690292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 581/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:52:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:52:04.713194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12d250-2a6e-4299-baef-05f9a0404b1c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:04.811610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12d250-2a6e-4299-baef-05f9a0404b1c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:04.844598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:04.845154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:04.845431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:04.845699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:04.848251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:04.857174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:52:04.861878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:52:04.862954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:52:04.864084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:52:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:52:04.869097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:04.893411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:04.969613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21b3f3-ba77-417b-8c2e-24cba37aa16f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:05.002741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:05.003340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:05.003637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:05.003913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:05.004569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:52:05.004870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:05.006232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:05.014571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 582/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:52:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:05.020960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:05.134682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98120-b482-42e4-b135-8b579f2447e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:05.169938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:05.170570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:05.170845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:05.171235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] PATH_INFO: `/v3/users/b9651653ae944a58bcf35550fff3701f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:05.185493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] RBAC: Authorizing `identity:get_user(user_id=b9651653ae944a58bcf35550fff3701f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:05.186951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:05.188631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:52:05 2022] GET /identity/v3/users/b9651653ae944a58bcf35550fff3701f => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:05.196238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:05.293400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:05.313723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4492-73db-4df8-abb0-4b5565a6c921 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:05.361499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:05.362094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:05.362370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:05.362638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:05.439559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:05.441360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:05.529638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:05.545901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 583/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:52:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:52:05.551883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:05.618909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:05.673931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08c8fe-76f1-4a9d-8348-38c832dd7489 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:05.715146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c39ad-5765-410c-b554-8d8a3ca28403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:05.716192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c39ad-5765-410c-b554-8d8a3ca28403 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:05.716698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c39ad-5765-410c-b554-8d8a3ca28403 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:05.717204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c39ad-5765-410c-b554-8d8a3ca28403 None admin] PATH_INFO: `/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:05.719154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c39ad-5765-410c-b554-8d8a3ca28403 None admin] RBAC: Authorizing `identity:get_project(project_id=70a921ca1bfe4098a4ec56ae8022f2b6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:05.721844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c39ad-5765-410c-b554-8d8a3ca28403 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:05.724547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:52:05 2022] GET /identity/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6 => generated 361 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:05.730658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:05.834266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84bf24-4544-460e-82f4-2b75cbfdd5d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:05.868511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:05.869120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:05.869399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:05.869672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:05.870390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=70a921ca1bfe4098a4ec56ae8022f2b6, user.id=b9651653ae944a58bcf35550fff3701f) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:52:05.870696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:05.872154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ee9a-e34d-41fb-bdd3-46760affcfa0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:05.877666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 584/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:52:05 2022] GET /identity/v3/role_assignments?user.id=b9651653ae944a58bcf35550fff3701f&scope.project.id=70a921ca1bfe4098a4ec56ae8022f2b6&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:07.840527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475e9f9-3055-43be-b6e7-ce3c5df9636f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:07.841072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475e9f9-3055-43be-b6e7-ce3c5df9636f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:07.841072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475e9f9-3055-43be-b6e7-ce3c5df9636f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:07.842410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1186] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:52:07.847577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:07.847978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:07.848145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:07.866697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:52:07.868089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:52:07.912283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:07.933676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 585/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:52:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:52:07.956316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475e9f9-3055-43be-b6e7-ce3c5df9636f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:08.053972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475e9f9-3055-43be-b6e7-ce3c5df9636f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:08.076944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:08.088767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:08.089379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:08.089654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:08.089921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:08.099552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:52:08.104454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:52:08.105526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:52:08.106361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:52:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:52:08.111398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:08.208671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1aa21-00b0-4bcb-bc20-f4e246c009c0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:08.241784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:08.242385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:08.242667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:08.242940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:08.243634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:52:08.243928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:08.245286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:08.253332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 586/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:08.259829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:08.356425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd74e29-48aa-4d5a-a68b-d6530fd72faf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:08.389262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:08.389874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:08.390153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:08.390424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] PATH_INFO: `/v3/users/b9651653ae944a58bcf35550fff3701f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:08.391539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] RBAC: Authorizing `identity:get_user(user_id=b9651653ae944a58bcf35550fff3701f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:08.392968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:08.394519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:52:08 2022] GET /identity/v3/users/b9651653ae944a58bcf35550fff3701f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:08.400766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:08.442889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:08.498365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102179d4-76b5-4379-85db-e66a6024a65d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:08.532538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f252-7724-4b2e-8bbc-44d0ab6371fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:08.533166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f252-7724-4b2e-8bbc-44d0ab6371fb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:08.533448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f252-7724-4b2e-8bbc-44d0ab6371fb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:08.533720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f252-7724-4b2e-8bbc-44d0ab6371fb None admin] PATH_INFO: `/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:08.534857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f252-7724-4b2e-8bbc-44d0ab6371fb None admin] RBAC: Authorizing `identity:get_project(project_id=70a921ca1bfe4098a4ec56ae8022f2b6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:08.536347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f252-7724-4b2e-8bbc-44d0ab6371fb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:08.537964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 587/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:52:08 2022] GET /identity/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:08.543941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:08.592659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:08.641011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b54536-1f5e-417e-881c-8a2fd0bd73d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:08.673413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227081e-b51b-456d-8a5e-8642eab82dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:08.674060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227081e-b51b-456d-8a5e-8642eab82dc7 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:08.674340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227081e-b51b-456d-8a5e-8642eab82dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:08.674613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227081e-b51b-456d-8a5e-8642eab82dc7 None admin] PATH_INFO: `/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6/users/b9651653ae944a58bcf35550fff3701f/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:08.676518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227081e-b51b-456d-8a5e-8642eab82dc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a921ca1bfe4098a4ec56ae8022f2b6, user_id=b9651653ae944a58bcf35550fff3701f, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:08.677931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227081e-b51b-456d-8a5e-8642eab82dc7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:08.691273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:52:08 2022] PUT /identity/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6/users/b9651653ae944a58bcf35550fff3701f/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:52:10.653635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8accc-c8aa-47ae-b40e-206af069391e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:10.654174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8accc-c8aa-47ae-b40e-206af069391e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:10.654174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8accc-c8aa-47ae-b40e-206af069391e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:10.655517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 588/1193] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:52:10.660664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:10.661109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:10.661203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:10.680803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:52:10.681974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:52:10.746826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:52:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:52:10.770108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8accc-c8aa-47ae-b40e-206af069391e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:10.871466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8accc-c8aa-47ae-b40e-206af069391e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:10.881683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:10.905281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:10.905836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:10.906112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:10.906382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:10.916519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:52:10.921086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:52:10.922177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:52:10.923216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 589/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:52:10.928174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:10.968105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:11.025619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d1b37-0840-479f-af25-faf7a4778da2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:11.057774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:11.058366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:11.058643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:11.058912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:11.059567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:52:11.059899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:11.061249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:11.068901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:11.075380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:11.170839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6a4-de25-4ce2-93ad-b1114f4cdc80 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:11.203487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:11.204134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:11.204416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:11.204690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] PATH_INFO: `/v3/users/b9651653ae944a58bcf35550fff3701f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:11.205794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] RBAC: Authorizing `identity:get_user(user_id=b9651653ae944a58bcf35550fff3701f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:11.207214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:11.208804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 590/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 9 06:52:11 2022] GET /identity/v3/users/b9651653ae944a58bcf35550fff3701f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:11.215087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:11.314449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8971-8837-4256-b3e0-4917b769e433 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:11.316009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:11.347688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:11.348288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:11.348596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:11.348870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:11.414106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:11.415637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:11.509941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:52:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:52:11.515758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:11.555324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:11.613885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd30e5c-b825-4095-8a25-8783cb1c2946 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:11.645732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a998846-e825-4eba-9e5d-6d848dfc777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:11.646348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a998846-e825-4eba-9e5d-6d848dfc777c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:11.646627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a998846-e825-4eba-9e5d-6d848dfc777c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:11.646900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a998846-e825-4eba-9e5d-6d848dfc777c None admin] PATH_INFO: `/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:11.648563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a998846-e825-4eba-9e5d-6d848dfc777c None admin] RBAC: Authorizing `identity:get_project(project_id=70a921ca1bfe4098a4ec56ae8022f2b6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:11.649969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a998846-e825-4eba-9e5d-6d848dfc777c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:11.651572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 591/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 9 06:52:11 2022] GET /identity/v3/projects/70a921ca1bfe4098a4ec56ae8022f2b6 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:11.657512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:52:11.659667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:11.753947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825403-a306-4b9a-8e62-6449319a8df0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:52:11.786697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:52:11.787229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:52:11.787507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:52:11.787808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:52:11.789247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=70a921ca1bfe4098a4ec56ae8022f2b6, user.id=b9651653ae944a58bcf35550fff3701f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:52:11.789557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:52:11.790962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3a9c8-eec3-4b7c-ae46-299fc1000067 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:52:11.796539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:52:11 2022] GET /identity/v3/role_assignments?user.id=b9651653ae944a58bcf35550fff3701f&scope.project.id=70a921ca1bfe4098a4ec56ae8022f2b6&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:52:21.641702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=47 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:52:21.794019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=67 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:06.410474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b59f4-aa5f-43b1-8735-0661c3e0258f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:06.411895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b59f4-aa5f-43b1-8735-0661c3e0258f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:06.412341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b59f4-aa5f-43b1-8735-0661c3e0258f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:06.414233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 592/1201] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:06.423660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:06.423660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:06.423660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:06.448959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:06.450200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:06.545836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:06.578104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b59f4-aa5f-43b1-8735-0661c3e0258f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:06.743253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b59f4-aa5f-43b1-8735-0661c3e0258f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:06.797437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:06.798545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:06.799105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:06.799530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:06.818003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:06.820831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:06.841841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:06.843246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:06.844344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:06.844344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:06.844344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:06.844344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 593/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:06.849675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:06.999494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648957d-9bd0-449a-8eac-aaa107714a99 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:07.050580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:07.057160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:07.059567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:07.059567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:07.059567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:07.059567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:07.059567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:07.060821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13645e7a-d810-4ba3-9647-20faf5ad501d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:07.075680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:53:07.081180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:07.147992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:07.207190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7ce07-2a4d-466e-8b3b-630929d208d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:07.246826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b48d-f640-414f-b044-253ffb4e08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:07.247430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b48d-f640-414f-b044-253ffb4e08c0 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:07.247740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b48d-f640-414f-b044-253ffb4e08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:07.248013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b48d-f640-414f-b044-253ffb4e08c0 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:07.248653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b48d-f640-414f-b044-253ffb4e08c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:07.250121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b48d-f640-414f-b044-253ffb4e08c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:07.279329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 594/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 9 06:53:07 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 06:53:09.541598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6633d60-3d86-42e0-b58c-53db378ce645 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:09.542143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6633d60-3d86-42e0-b58c-53db378ce645 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:09.542143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6633d60-3d86-42e0-b58c-53db378ce645 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:09.543487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1206] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:09.549184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:09.549689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:09.549805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:09.569034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:09.570410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:09.648558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 595/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:09.672290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6633d60-3d86-42e0-b58c-53db378ce645 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:09.806008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6633d60-3d86-42e0-b58c-53db378ce645 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:09.850389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:09.851190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:09.851630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:09.852432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:09.869578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:53:09.874505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:09.875527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:09.876727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:53:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:53:09.884191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:09.934957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:10.013221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec219a6-6171-41e2-8b3d-a4d73699db02 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:10.055395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:10.056186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:10.056334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:10.056606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:10.057274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:10.057576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:10.058951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:10.073096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 596/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:53:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:10.074180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:10.128409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:10.188498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a57828-032b-40df-a37c-b6a657de4e6d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:10.235157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:10.235910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:10.237571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:10.237571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:10.248940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:10.250395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:10.268333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:10.269358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:10.270947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:10.270947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:10.270947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:10.270947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:10.276052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:10.412141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:10.416291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487337bc-8a37-4726-89e4-ea04bcabf8d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:10.460580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:10.461240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:10.461567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:10.461924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:10.462758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:10.463057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:10.464694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:10.481507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 597/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:53:10.488148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:10.633223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea86a68-ec80-4971-a1c2-e9524b130ed2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:10.634836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:10.675315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:10.676011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:10.676311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:10.676585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:10.685278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:10.686835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:53:10.699248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:10.700813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:10.701751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:10.706410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:10.829155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4b5c3-5293-4373-909a-b3f01d2a0300 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:10.873946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:10.874662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:10.874942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:10.875259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:10.876358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:10.876549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:10.877974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:10.898242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 598/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 9 06:53:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:10.904064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:11.018956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1310c3-dca3-44b3-8386-f0b471f9114d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:11.057111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:11.057750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:11.058042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:11.058398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:11.091566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:11.136182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:11.137720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:11.263630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:53:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:53:11.267018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:11.292966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:11.377680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb11de0-bc6a-4400-bb12-b6cc0bae8bf6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:11.416390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:11.417034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:11.417320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:11.417594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:11.429465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:11.431027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:11.447835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:11.449197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:11.450244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:11.450244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:11.450244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:11.450244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 599/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:11.455193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:11.524989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:11.576271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e3303-7ca3-4505-a6dc-9d6bc75c18ae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:11.615706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:11.616600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:11.617040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:11.617405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:11.618138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:11.618601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:11.620381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:11.633772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:11.640564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:11.729311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:11.762048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a86940-dc22-4f9b-9afe-18f618f85f25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:11.797028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:11.797807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:11.798189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:11.798598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:11.806462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:11.808079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:53:11.821267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:11.822443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:11.823568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:11.825064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 600/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:53:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:11.831221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:11.945299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d940-9c8b-457a-8cc9-2c96b6b69350 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:11.968867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:11.982783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:11.983510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:11.983937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:11.984319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:11.985123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:11.985522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:11.987072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d061a6-bd35-4ae7-a248-46eb12a9f0f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:11.999770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:53:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:12.006597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:12.110801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed43567-fb21-4308-b891-b9f734fac2c0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:12.142054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:12.148956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:12.149598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:12.149889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:12.150237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:12.150971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=86a917f8f79a494bbd8c18fbefe695ba) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:12.151285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:12.152920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b6347-73c2-4281-b80d-11f1564ee904 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:12.158730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 601/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:53:12 2022] GET /identity/v3/role_assignments?user.id=86a917f8f79a494bbd8c18fbefe695ba&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:14.212508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3cae-0f2e-47f7-9807-b0e5a383114d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:14.213284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3cae-0f2e-47f7-9807-b0e5a383114d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:14.213284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3cae-0f2e-47f7-9807-b0e5a383114d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:14.214402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1220] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:14.219624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:14.220075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:14.220372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:14.239319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:14.240806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:14.319168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 602/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:14.341241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3cae-0f2e-47f7-9807-b0e5a383114d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:14.454302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3cae-0f2e-47f7-9807-b0e5a383114d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:14.489948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:14.490500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:14.490777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:14.491047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:14.501463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:53:14.506782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:14.507973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:14.509174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:53:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:14.513900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:14.630116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2cc20-1d9a-46ab-b1e7-40fe2f0170a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:14.654755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:14.666503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:14.667115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:14.667399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:14.667710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:14.668418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:14.668695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:14.670059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:14.678784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 603/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:53:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:14.684817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:14.729736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:14.790091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69143-fdb5-453e-bde1-c7a9456665e1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:14.827036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:14.827905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:14.828220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:14.828508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:14.843036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:14.844707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:14.864164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:14.865518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:14.866612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:14.866612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:14.866612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:14.866612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:14.871537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:14.986318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7970a-190f-4637-b435-f9319f4fa5a0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:15.020153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:15.020759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:15.021036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:15.021311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:15.021951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:15.022252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:15.023637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:15.037122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 604/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:15.042981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:15.142052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:15.149155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c585b5-12f3-41db-8e79-c2f1bb1e2179 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:15.183283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:15.183963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:15.184228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:15.184498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:15.192449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:15.193996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:53:15.205669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:15.206850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:15.208609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:15.215277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:15.267352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:15.325746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d660c1d-09ba-40c2-8370-233a7d2f5eef None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:15.361937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:15.362545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:15.362883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:15.363093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:15.364760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:15.365071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:15.366629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:15.387775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 605/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 9 06:53:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:15.393803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:15.514875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb14ad-b0f7-4237-99aa-5bc8215f3a92 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:15.560044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:15.560735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:15.561055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:15.561380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:15.573782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:15.575399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:15.591395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:15.597369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:15.598975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:15.598975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:15.598975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:15.598975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:53:15.599405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:15.706557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ed8bd-36a1-467c-be78-4d53b18606eb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:15.709779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:15.743562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:15.744246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:15.744525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:15.744845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:15.745609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:15.745934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:15.747302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:15.760023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 606/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:53:15.766326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:15.777026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:15.888220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd600-08a5-47be-aee5-63ce91ede249 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:15.925590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:15.927004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:15.927355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:15.927696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:15.935219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:15.936913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:53:15.949199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:15.950198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:15.951086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:53:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:53:15.956634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:16.067285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ea00c-dfd2-4322-804f-50e056e10630 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:16.103355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:16.103998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:16.104276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:16.104550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:16.105240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:16.105568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:16.106999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b5646-7e1f-478e-a8a7-a81e4f8887f3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:16.120642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 607/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:53:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:16.126887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:16.198035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:16.234359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3fdd-fed8-41b1-bb75-ef4d5b993967 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:16.269466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ed78e-856c-4f2c-bdac-a389c5f38e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:16.270742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed78e-856c-4f2c-bdac-a389c5f38e70 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:16.271022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed78e-856c-4f2c-bdac-a389c5f38e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:16.271294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed78e-856c-4f2c-bdac-a389c5f38e70 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/86a917f8f79a494bbd8c18fbefe695ba/roles/319875e1dbf740f89d20e2ae082de786` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:16.286929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ed78e-856c-4f2c-bdac-a389c5f38e70 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=86a917f8f79a494bbd8c18fbefe695ba, role_id=319875e1dbf740f89d20e2ae082de786)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:16.288425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ed78e-856c-4f2c-bdac-a389c5f38e70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:16.302622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:53:16 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/86a917f8f79a494bbd8c18fbefe695ba/roles/319875e1dbf740f89d20e2ae082de786 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:53:18.616571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3e06c-fce1-41ff-9c5a-48a69308b454 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:18.617184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3e06c-fce1-41ff-9c5a-48a69308b454 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:18.617184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3e06c-fce1-41ff-9c5a-48a69308b454 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:18.618436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 608/1233] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:18.623697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:18.624337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:18.624442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:18.643714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:18.645770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:18.716123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:18.744411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3e06c-fce1-41ff-9c5a-48a69308b454 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:18.773609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:18.864373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3e06c-fce1-41ff-9c5a-48a69308b454 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:18.898841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:18.899470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:18.899772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:18.900045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:18.912919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:53:18.917885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:18.919189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:18.920340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 609/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 9 06:53:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:53:18.928514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:18.982220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:19.046055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025f9c6-eb6c-4008-a294-07801fb832c2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:19.085513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:19.086150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:19.086410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:19.086679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:19.087432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:19.087801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:19.089295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:19.104449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 9 06:53:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:53:19.105887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:19.193517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:19.210297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd05a1-b7ab-4da9-8c4e-28a22cab6080 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:19.248400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:19.249023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:19.249299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:19.249568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:19.260988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:19.262453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:19.277148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:19.278392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:19.285036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:19.285036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:19.285036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:19.285036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 610/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:19.285586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:19.387907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b7729-53f5-424d-b821-378d7b651c87 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:19.397734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:19.425866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:19.426572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:19.426926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:19.427273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:19.428026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:19.428467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:19.430267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:19.443198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:19.449331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:19.562407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee8ac9-c986-413c-b4d5-4f2e359fc345 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:19.603981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:19.604673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:19.605056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:19.605326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:19.608077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:19.614489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:19.616076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:53:19.627937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:19.630002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:19.631250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 611/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:19.636597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:19.752074 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cbfd2-1954-4a50-9d33-28d16640e3b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:19.787870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:19.788492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:19.788786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:19.789054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:19.789782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:19.790093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:19.791525 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:19.814252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 9 06:53:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:19.815513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:19.929802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab7423-3238-45a6-9a33-5f174c714859 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:19.965573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:19.966184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:19.966471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:19.966748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:20.034315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:20.042656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:20.044211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:20.150576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 612/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:53:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:53:20.156312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:20.184469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:20.265615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294bfa7-6a11-41e0-8b7f-438f50ec16cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:20.300954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:20.301517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:20.301795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:20.302081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:20.312954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:20.314408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:20.331573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:20.333481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:20.334460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:20.334460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:20.334460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:20.334460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:20.339547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:20.443027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:20.450046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b5bc-41c6-448c-881f-fd927b13f806 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:20.488636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:20.489449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:20.489778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:20.490056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:20.490848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:20.491166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:20.492583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:20.505748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 613/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:20.512004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:20.609307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:20.636775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e402f6-2caf-4c34-9cca-5f4a7ef72972 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:20.672144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:20.672877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:20.673164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:20.673475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:20.681479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:20.682911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:53:20.695924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:20.697127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:20.698383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:53:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:20.703581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:20.820784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00387a-0c0b-44d9-acd5-d3a641708c15 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:20.859998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:20.860561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:20.860836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:20.861109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:20.861806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:20.862111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:20.864225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613efa8-0e50-48c7-ac1d-90d587765196 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:20.876404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 614/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:20.883268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:20.995920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712690a-424a-488f-bb30-86fc9f51047a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:21.033017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:21.033700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:21.033939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:21.034236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:21.035002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] RBAC: Adding query filter params (role.id=319875e1dbf740f89d20e2ae082de786, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=86a917f8f79a494bbd8c18fbefe695ba) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:21.035355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:21.036898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb87d4-c215-4fd7-bc17-eabcb62aa27e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:21.044115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:53:20 2022] GET /identity/v3/role_assignments?user.id=86a917f8f79a494bbd8c18fbefe695ba&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=319875e1dbf740f89d20e2ae082de786 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:23.199882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71facfc5-8509-4ea3-8ebd-39f398a4071a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:23.200440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71facfc5-8509-4ea3-8ebd-39f398a4071a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:23.200440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71facfc5-8509-4ea3-8ebd-39f398a4071a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:23.201833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 615/1247] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:23.208265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:23.208785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:23.208893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:23.215359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:23.228854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:23.230082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:23.306867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:23.330779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71facfc5-8509-4ea3-8ebd-39f398a4071a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:23.352149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:23.446734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71facfc5-8509-4ea3-8ebd-39f398a4071a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:23.485609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:23.486168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:23.486467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:23.486725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:23.499195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:53:23.504712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:23.505883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:23.511924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 616/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:53:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:23.512796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:23.641040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9cfe8-782b-49d8-ae0c-7364336e1dc7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:23.678450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:23.684117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:23.684735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:23.685013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:23.685281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:23.685937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:23.686239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:23.687589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:23.696859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:53:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:23.702555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:23.792512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:23.820777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c618b6-c3d1-46ae-b5d6-9aa735a91828 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:23.861661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:23.862304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:23.862590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:23.863445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:23.874841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:23.876597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:23.893114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:23.899548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:23.900553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:23.900553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:23.900553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:23.900553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 617/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:23.900832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:24.020869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3899bc-dac7-4479-b8cf-9891f7e2b139 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:24.059583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:24.060162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:24.060456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:24.060697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:24.061371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:24.061672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:24.063050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:24.077738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:24.083941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:24.202002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b6a1-f1d1-4620-97fc-57b5fbfad4e5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:24.231038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:24.240433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:24.241170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:24.241456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:24.241735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:24.250542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:24.252445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:53:24.264994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:24.266233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:24.268887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 618/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 06:53:24.272129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:24.334549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:24.386761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57b43e-3471-4da5-97cb-26b37929ed4d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:24.423922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:24.424530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:24.424838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:24.425112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:24.426432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:24.426771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:24.428246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:24.449616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 9 06:53:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:53:24.456097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:24.573608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f00ded-c97c-458f-9de2-651bde88b3b8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:24.611928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:24.612461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:24.613277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:24.613558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:24.671203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:24.684275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:24.685718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:24.793441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 619/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:53:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:53:24.799446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:24.924289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8e106-7e9a-4e9c-8e2b-a6f1dc050aee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:24.939760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:24.962086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:24.962749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:24.963032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:24.963305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:24.975975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:24.977545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:25.003345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:25.004509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:25.005648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:25.005648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:25.005648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:25.005648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:25.011762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:25.106670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:25.119701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5f4d2-91f2-4426-8b57-d3d9d71a54db None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:25.157723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:25.158360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:25.158639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:25.158911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:25.159546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:25.160428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:25.161782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:25.172933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 620/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:25.179298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:25.290440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7661df-b2e6-4357-a5f6-3619a7906850 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:25.325425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:25.326061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:25.326349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:25.326620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:25.334113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:25.336183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:53:25.348331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:25.349426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:25.350512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:25.355422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:25.464547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b4906-508b-48d9-a35a-643ff011088a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:25.498738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:25.499340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:25.499638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:25.499914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:25.500598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:25.501162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:25.502595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bce448-0596-4240-97a1-a8fa6756f9fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:25.515293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 621/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:25.522108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:25.552138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:25.639455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8b5f-0a87-4409-9222-904c25620ea6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:25.674635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:25.675278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:25.675756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:25.676037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:25.676758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=86a917f8f79a494bbd8c18fbefe695ba) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:25.677069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:25.678677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde4011-4aef-4539-a243-82b25c4adade None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:25.684810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:53:25 2022] GET /identity/v3/role_assignments?user.id=86a917f8f79a494bbd8c18fbefe695ba&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=d58e69f2b822466daca66a9696521a24 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:27.677786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd44e8-2ab1-4141-bdff-4a592c7798f8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:27.678342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd44e8-2ab1-4141-bdff-4a592c7798f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:27.678342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd44e8-2ab1-4141-bdff-4a592c7798f8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:27.679685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 622/1261] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:27.684597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:27.684972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:27.685150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:27.704048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:27.705216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:27.768591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:27.790557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd44e8-2ab1-4141-bdff-4a592c7798f8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:27.801842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:27.893149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd44e8-2ab1-4141-bdff-4a592c7798f8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:27.926332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:27.927039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:27.927318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:27.927586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:27.937464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:53:27.942123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:27.943262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:27.944277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 623/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:53:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:27.949205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:27.966776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:28.044367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8d80-715e-409c-958a-098d25095b89 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:28.076995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:28.077590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:28.077865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:28.078132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:28.078788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:28.079086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:28.080467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:28.088958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:53:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:28.094759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:28.159217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:28.192247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9859-2d31-4457-be04-18ec758365c6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:28.224907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:28.225513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:28.225787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:28.226055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:28.235744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:28.237162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:28.251393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:28.252512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:28.253453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:28.253453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:28.253453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:28.253453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 624/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:28.258473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:28.337188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:28.356312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aa89a-4f26-4bd7-9600-ffb3601da558 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:28.388050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:28.388638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:28.388914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:28.389184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:28.389810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:28.390109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:28.391479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:28.403398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:28.409116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:28.507297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aeb20-10a5-4363-9c91-882782983d0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:28.526165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:28.541646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:28.542252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:28.542568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:28.543024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:28.553164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:28.554920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:53:28.568420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:28.569565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:28.570738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 625/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:28.576507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:28.681303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a3700-a38f-41e3-80ab-f4062240c0fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:28.713429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:28.714008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:28.714286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:28.714552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:28.715261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:28.715563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:28.716991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:28.726037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:28.734324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 9 06:53:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:28.740604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:28.851987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0f04c-2ce7-41ee-88b5-d9cf7609da92 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:28.885356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:28.885984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:28.886263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:28.886536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:28.896636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:28.898053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:28.911505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:28.912647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:28.913878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:28.913878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:28.913878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:28.913878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 626/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:28.918226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:29.015012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7003fce-4ed6-41c7-a910-83849508c278 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:29.047768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:29.048370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:29.048650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:29.048923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:29.049558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:29.049859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:29.051808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:29.062967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:29.069209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:29.101132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:29.168258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6837d70-9c58-40ed-89c0-928bc764b848 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:29.200379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:29.200995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:29.201274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:29.201544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:29.208657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:29.210072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:53:29.220898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:29.227241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:29.228413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 627/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:53:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:29.229057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:29.285447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:29.325026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55082c7-4c8b-42f1-a661-ae72df419d5d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:29.356697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:29.357287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:29.357563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:29.357830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:29.358505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:29.358806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:29.360255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2cf6-f9f7-4cd0-9d91-2d9f4af8223e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:29.371616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:53:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:29.377690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:29.466716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:29.475576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093075b-70d7-4dd2-af0a-88b5751a5093 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:29.508789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0de769-4b61-441e-a35b-f16c2d6f4a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:29.509429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0de769-4b61-441e-a35b-f16c2d6f4a64 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:29.509705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0de769-4b61-441e-a35b-f16c2d6f4a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:29.509976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0de769-4b61-441e-a35b-f16c2d6f4a64 None admin] PATH_INFO: `/v3/projects/2051c000cdba46208ac96365247f6b94/users/86a917f8f79a494bbd8c18fbefe695ba/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:29.511833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0de769-4b61-441e-a35b-f16c2d6f4a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051c000cdba46208ac96365247f6b94, user_id=86a917f8f79a494bbd8c18fbefe695ba, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:29.513239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0de769-4b61-441e-a35b-f16c2d6f4a64 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:29.525598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 628/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 9 06:53:29 2022] PUT /identity/v3/projects/2051c000cdba46208ac96365247f6b94/users/86a917f8f79a494bbd8c18fbefe695ba/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:53:31.478892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d032a-b927-491d-9a9d-6b01907bfcda None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:31.479442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d032a-b927-491d-9a9d-6b01907bfcda None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:31.479442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d032a-b927-491d-9a9d-6b01907bfcda None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:31.480802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1274] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:31.485748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:31.486141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:31.486305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:31.505158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:31.506338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:31.577390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 629/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:31.606057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d032a-b927-491d-9a9d-6b01907bfcda None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:31.705929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d032a-b927-491d-9a9d-6b01907bfcda None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:31.722892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:31.740012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:31.740569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:31.740849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:31.741119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:31.751063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 06:53:31.756542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:31.757623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:31.758490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 9 06:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:31.763677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:31.841873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:31.860962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf85eb-0adf-4db6-8d8a-9bf4ce0bfe53 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:31.895417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:31.896048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:31.896325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:31.896592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:31.897245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:31.897542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:31.898893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:31.906555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 630/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 9 06:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:31.912510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:32.010255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe98a1-7636-4c80-9a59-fe8819520ea9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:32.042113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:32.042722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:32.042996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:32.043263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:32.053467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:32.054883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:32.069395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:32.070500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:32.071391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:32.071391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:32.071391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:32.071391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:32.076261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:32.171417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75374388-e535-44e2-adaf-86a94b9b2c9c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:32.203796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:32.204385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:32.204659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:32.204929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:32.205560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:32.205856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:32.207283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:32.218372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 631/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:32.224504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:32.255656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:32.323451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f3155-51d0-4451-a7b5-7275fdd8ecf7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:32.356075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:32.356686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:32.356965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:32.357239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:32.364547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:32.365969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 06:53:32.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:32.379209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:32.380325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:32.384455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:32.388186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:32.481987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef87f8-5b09-4346-9543-a44330c6ca18 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:32.515347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:32.515967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:32.516242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:32.516514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:32.517231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:32.517533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:32.519028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:32.536377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 632/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 9 06:53:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:32.542656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:32.637996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:32.648294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f104c-5047-477e-b61a-968a60c7b213 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:32.680705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:32.681303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:32.681581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:32.681851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:32.746091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:32.747544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:32.832995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:32.843473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 9 06:53:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 06:53:32.849753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:32.910547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:32.947701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde07875-78a7-410a-b77d-4909d73a846d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:32.982288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:32.982916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:32.983198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:32.983471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:32.993224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:32.994664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 06:53:33.009512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:33.010630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 06:53:33.011566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 06:53:33.011566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 06:53:33.011566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:33.011566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 633/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:33.016260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:33.112342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964109df-6521-4feb-b3fe-8de4f968670a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:33.146627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:33.147224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:33.147504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:33.147811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:33.148447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:33.148747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:33.150104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:33.161011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 9 06:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:33.166512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:33.263264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce044e50-2a46-448f-a06c-7f4896d1e8f2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:33.276775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:33.296184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:33.296797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:33.297075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:33.297347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:33.304177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:33.305584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 06:53:33.316921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:33.317978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Aug 09 06:53:33.319223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 634/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 9 06:53:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:33.323711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:33.373443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:33.422839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86027036-97bf-4504-95d7-23663e64cc7e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:33.457487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:33.458079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:33.458357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:33.458630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:33.459330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:33.459633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:33.461043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e984730-f15d-47ab-901b-4d9c82575689 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:33.473470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 9 06:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:33.479502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:33.581648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eff0bd-3f3b-4ec4-a3a4-7a6e3bf87737 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:33.614296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:33.614886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:33.615166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:33.615439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:33.616188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] RBAC: Adding query filter params (role.id=d58e69f2b822466daca66a9696521a24, scope.project.id=2051c000cdba46208ac96365247f6b94, user.id=86a917f8f79a494bbd8c18fbefe695ba) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:33.616495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:33.617920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b8f8e-f0c3-4bbb-bd84-a7ce23a40e08 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:33.623485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 635/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 9 06:53:33 2022] GET /identity/v3/role_assignments?user.id=86a917f8f79a494bbd8c18fbefe695ba&scope.project.id=2051c000cdba46208ac96365247f6b94&role.id=d58e69f2b822466daca66a9696521a24 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:35.627036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3eda3-0671-4f53-9027-4b40a31c66d2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:35.627591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3eda3-0671-4f53-9027-4b40a31c66d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:35.627591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3eda3-0671-4f53-9027-4b40a31c66d2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:35.628963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1288] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:35.633953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:35.634354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:35.634532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:35.652665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:35.653847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:35.683220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:35.718875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 636/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:35.741223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3eda3-0671-4f53-9027-4b40a31c66d2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:35.838479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3eda3-0671-4f53-9027-4b40a31c66d2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:35.848967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:35.871921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:35.872541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:35.872816 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:35.873083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:35.873656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:35.875012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:35.884252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:35.885550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:35.886687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:53:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:35.891144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:35.986100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d084-d1dc-41fe-8ef1-39ca3182a59e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:36.018966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:36.019558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:36.019864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:36.020139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:36.020759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:36.021067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:36.022418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f62698-3670-4caa-ab9b-ea94fb7750ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:36.027040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 637/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 9 06:53:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:36.032787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:36.131336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0f7a4-13ca-41ee-9587-e8ca2c9044a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:36.163879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:36.164470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:36.164752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:36.165027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:36.165653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:36.165958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:36.167305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660686d5-da3c-4d09-8cd6-16e0417cd0e6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:36.171578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 9 06:53:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:38.154929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73325324-7ead-47ce-b7fb-6d641f04f107 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:38.155430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73325324-7ead-47ce-b7fb-6d641f04f107 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:38.155544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73325324-7ead-47ce-b7fb-6d641f04f107 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:38.156852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 638/1293] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:38.161963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4dbf2-6e37-479b-8693-105f6be00e9b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:38.162421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4dbf2-6e37-479b-8693-105f6be00e9b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:38.162511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4dbf2-6e37-479b-8693-105f6be00e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:38.180658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8ad4dbf2-6e37-479b-8693-105f6be00e9b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:38.181839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ad4dbf2-6e37-479b-8693-105f6be00e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:38.210521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:38.248476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:38.271377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73325324-7ead-47ce-b7fb-6d641f04f107 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:38.302429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:38.370772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73325324-7ead-47ce-b7fb-6d641f04f107 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:38.403281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaefe55-e8e1-4b8f-a1c1-b0e053797eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:38.403903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaefe55-e8e1-4b8f-a1c1-b0e053797eb0 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:38.404182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaefe55-e8e1-4b8f-a1c1-b0e053797eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:38.404455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaefe55-e8e1-4b8f-a1c1-b0e053797eb0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:38.405148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaefe55-e8e1-4b8f-a1c1-b0e053797eb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:38.406449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaefe55-e8e1-4b8f-a1c1-b0e053797eb0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:38.420820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 639/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 9 06:53:38 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:53:40.397925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a9828-3386-46a1-a019-b81daa0659d5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:40.398454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a9828-3386-46a1-a019-b81daa0659d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:40.398454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a9828-3386-46a1-a019-b81daa0659d5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:40.399874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:40.404931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:40.405347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:40.405480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:40.423828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:40.425113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:40.495449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 640/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:40.519895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a9828-3386-46a1-a019-b81daa0659d5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:40.629148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a9828-3386-46a1-a019-b81daa0659d5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:40.661773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:40.662326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:40.662604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:40.662874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:40.663447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:40.664830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:40.673199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:40.674243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:40.675727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:53:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:40.680715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:40.698954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:40.780105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ada74-8af9-4a25-a47c-1be281f26d7d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:40.812900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:40.813501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:40.813782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:40.814057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:40.814684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:40.815504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:40.816927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:40.821912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 641/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 9 06:53:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 06:53:40.827869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:40.859134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:40.927413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e4e7-0e0c-4ab8-b72a-1ef2c4b557bd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:40.960390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:40.960989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:40.961267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:40.961539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:40.962231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] RBAC: Adding query filter params (interface=public, service_id=cfe2c5c6ae394d808a1171d0dd9f147a, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:40.962554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:40.963915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3103b-c50a-4418-9886-93c3c63f52c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:40.968772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 9 06:53:40 2022] GET /identity/v3/endpoints?service_id=cfe2c5c6ae394d808a1171d0dd9f147a&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:40.974459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:41.055365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:41.073719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093ca4e-4e9f-41b6-b73d-089c546c745b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:41.108154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965559e5-1715-4760-82f1-5cb1215d6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:41.108754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965559e5-1715-4760-82f1-5cb1215d6b95 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:41.109035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965559e5-1715-4760-82f1-5cb1215d6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:41.109307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965559e5-1715-4760-82f1-5cb1215d6b95 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:41.109963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965559e5-1715-4760-82f1-5cb1215d6b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:41.111323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965559e5-1715-4760-82f1-5cb1215d6b95 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:41.118151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 642/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 9 06:53:40 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:53:43.077067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f1a8a-d47e-41dd-afe0-53ad8f304113 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:43.077589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f1a8a-d47e-41dd-afe0-53ad8f304113 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:43.077683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f1a8a-d47e-41dd-afe0-53ad8f304113 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:43.079245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1302] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:53:43.084989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:43.085432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:43.085538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:43.104170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:53:43.105523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:53:43.171974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 643/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:53:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 06:53:43.196284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f1a8a-d47e-41dd-afe0-53ad8f304113 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:43.287985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:43.294588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f1a8a-d47e-41dd-afe0-53ad8f304113 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:43.326549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:43.327165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:43.327444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:43.327749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:43.328327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:43.329673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 06:53:43.338733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 06:53:43.339914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 06:53:43.340764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 9 06:53:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 06:53:43.345862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:43.420685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:43.443580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3918-c7d0-45da-89f5-5d749a5621ec None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:43.477079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:43.477680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:43.477958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:43.478230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:43.478849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:53:43.479149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:43.480538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1036-1b47-4372-8315-235e38f5a826 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:43.485473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 644/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 9 06:53:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:53:43.491239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:53:43.594122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a38b74-3655-4843-8c4b-0c26baf8d3c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:53:43.626053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4452486-55b9-419e-87bb-c2f9eb0e689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:53:43.626650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4452486-55b9-419e-87bb-c2f9eb0e689d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:53:43.626927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4452486-55b9-419e-87bb-c2f9eb0e689d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:53:43.627199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4452486-55b9-419e-87bb-c2f9eb0e689d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:53:43.628499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4452486-55b9-419e-87bb-c2f9eb0e689d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:53:43.629847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4452486-55b9-419e-87bb-c2f9eb0e689d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:53:43.648778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:53:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:53:53.483028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=30 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:53:53.644944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:46.584756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319f6f-e946-4c9b-978e-57c74b43e954 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:46.585269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319f6f-e946-4c9b-978e-57c74b43e954 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:46.585269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319f6f-e946-4c9b-978e-57c74b43e954 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:46.586657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 645/1307] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:46.592590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb6898-4709-4658-a66b-998b26931c78 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:46.593568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb6898-4709-4658-a66b-998b26931c78 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:46.593842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb6898-4709-4658-a66b-998b26931c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:46.760747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76fb6898-4709-4658-a66b-998b26931c78 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:46.762137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76fb6898-4709-4658-a66b-998b26931c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:46.920207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:46.956271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5c1e-b5ef-45e8-a9ab-f39d36778275 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:46.956928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5c1e-b5ef-45e8-a9ab-f39d36778275 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:46.957419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5c1e-b5ef-45e8-a9ab-f39d36778275 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:46.959204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 646/1309] 10.222.0.30 () {54 vars in 965 bytes} [Tue Aug 9 06:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:46.965416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7d6d-0c13-44b4-967c-ca9781d6d65a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:46.965704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7d6d-0c13-44b4-967c-ca9781d6d65a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:46.965978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7d6d-0c13-44b4-967c-ca9781d6d65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:47.046609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-182a7d6d-0c13-44b4-967c-ca9781d6d65a None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:47.048000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-182a7d6d-0c13-44b4-967c-ca9781d6d65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:47.069524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:47.128019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Tue Aug 9 06:54:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:47.136935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5c1e-b5ef-45e8-a9ab-f39d36778275 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:54:47.139038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f3aa5c1e-b5ef-45e8-a9ab-f39d36778275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:47.274926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5c1e-b5ef-45e8-a9ab-f39d36778275 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:54:47.359535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:54:47.360179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:47.360462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:47.360739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:47.362397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:47.424985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:47.490203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:54:47.492830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e7cff-a0c6-46f7-9d18-e7d707190834 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:54:47.513979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 647/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:54:49.861367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c395d-9457-4ae4-a31a-3048534caa77 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:49.861812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c395d-9457-4ae4-a31a-3048534caa77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:49.861917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c395d-9457-4ae4-a31a-3048534caa77 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:49.863275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1312] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:49.868693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cb913-302e-4cd0-aab0-3e747dc30c51 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:49.869090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cb913-302e-4cd0-aab0-3e747dc30c51 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:49.869277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cb913-302e-4cd0-aab0-3e747dc30c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:49.943432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1d8cb913-302e-4cd0-aab0-3e747dc30c51 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:49.944776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1d8cb913-302e-4cd0-aab0-3e747dc30c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:50.049022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 648/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:50.088304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c395d-9457-4ae4-a31a-3048534caa77 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:54:50.089886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-046c395d-9457-4ae4-a31a-3048534caa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:50.194290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c395d-9457-4ae4-a31a-3048534caa77 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:54:50.271066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:50.273594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:54:50.274282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:50.274481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:50.275088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:50.277300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:50.388416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:54:50.390472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac69d8-2bd6-4021-99fa-00dfc4746e10 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:54:50.407843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:54:52.701675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1fa52-199d-4451-9b8d-e4325d3de248 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:52.702239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1fa52-199d-4451-9b8d-e4325d3de248 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:52.702353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1fa52-199d-4451-9b8d-e4325d3de248 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:52.703655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 649/1315] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:52.708850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020f6d-2773-40e0-9558-44e302f93e9c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:52.709248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020f6d-2773-40e0-9558-44e302f93e9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:52.709403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020f6d-2773-40e0-9558-44e302f93e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:52.773722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5a020f6d-2773-40e0-9558-44e302f93e9c None None] MFA Rules not processed for user `e5ba3c8cd91e4ef6b8737d677d96472d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:52.774925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a020f6d-2773-40e0-9558-44e302f93e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:52.825546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:52.870021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:54:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:52.925738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148731f-dde8-4438-b539-bdab201343ee None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:52.926020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148731f-dde8-4438-b539-bdab201343ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:52.926297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148731f-dde8-4438-b539-bdab201343ee None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:52.927670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 650/1317] 10.222.0.30 () {54 vars in 965 bytes} [Tue Aug 9 06:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:52.933615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93606bc-73e1-460a-a516-476eea10b22e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:52.933910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93606bc-73e1-460a-a516-476eea10b22e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:52.934193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93606bc-73e1-460a-a516-476eea10b22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:53.010127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b93606bc-73e1-460a-a516-476eea10b22e None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:53.011258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b93606bc-73e1-460a-a516-476eea10b22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:53.084381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1318] 10.222.0.30 () {58 vars in 1062 bytes} [Tue Aug 9 06:54:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:53.094154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148731f-dde8-4438-b539-bdab201343ee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:54:53.095721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f148731f-dde8-4438-b539-bdab201343ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:53.105338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:53.196201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148731f-dde8-4438-b539-bdab201343ee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:54:53.268190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:54:53.268720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:53.268996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:53.269263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:53.270842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:53.330970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:53.376708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:54:53.378131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a74368-16c8-4f2f-8fd5-1e4fd1816966 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:54:53.397624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 651/1319] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:54:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:54:56.298966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432e1c1-d5e3-4a6e-82ab-c06dec97df20 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:56.299527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432e1c1-d5e3-4a6e-82ab-c06dec97df20 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:56.299527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432e1c1-d5e3-4a6e-82ab-c06dec97df20 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:56.301879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1320] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:56.307076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9898fd2-86aa-4da9-93ee-4c02f744d1a6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:56.307471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9898fd2-86aa-4da9-93ee-4c02f744d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:56.307663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9898fd2-86aa-4da9-93ee-4c02f744d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:56.372776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f9898fd2-86aa-4da9-93ee-4c02f744d1a6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:56.374757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f9898fd2-86aa-4da9-93ee-4c02f744d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:56.468518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 652/1321] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:54:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:56.507686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432e1c1-d5e3-4a6e-82ab-c06dec97df20 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:54:56.509212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a432e1c1-d5e3-4a6e-82ab-c06dec97df20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:56.560104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:56.613981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432e1c1-d5e3-4a6e-82ab-c06dec97df20 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:54:56.685554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:54:56.686935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:56.687324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:56.687618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:56.689211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:56.782805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:56.796870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:54:56.798296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450514b-51ae-4fc8-b026-c612a7c3d2fa service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:54:56.816494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:54:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:54:59.490969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da382f-7f64-44e8-a1e8-fd5dce01af37 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:59.491501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da382f-7f64-44e8-a1e8-fd5dce01af37 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:59.491501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da382f-7f64-44e8-a1e8-fd5dce01af37 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:59.492958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 653/1323] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:54:59.498183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab54e0-ab53-48fe-8b2c-a117e6853202 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:59.499527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab54e0-ab53-48fe-8b2c-a117e6853202 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:59.499670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab54e0-ab53-48fe-8b2c-a117e6853202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:59.574221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-baab54e0-ab53-48fe-8b2c-a117e6853202 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:54:59.575427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-baab54e0-ab53-48fe-8b2c-a117e6853202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:59.679415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1324] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:54:59.717392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da382f-7f64-44e8-a1e8-fd5dce01af37 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:54:59.719234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-07da382f-7f64-44e8-a1e8-fd5dce01af37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:59.741646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:54:59.831183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da382f-7f64-44e8-a1e8-fd5dce01af37 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:54:59.907044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:54:59.907683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:54:59.907972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:54:59.908251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:54:59.909851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:54:59.981770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:00.021643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:00.022998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14a63a-df7b-4cb8-96d4-a484347d4d43 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:00.041885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 654/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:02.095959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121cb52-fc6e-430b-b2d6-931c6708e805 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:02.096510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121cb52-fc6e-430b-b2d6-931c6708e805 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:02.096510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121cb52-fc6e-430b-b2d6-931c6708e805 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:02.097845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1326] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:02.103099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316207b3-727b-4372-b98f-cf8d6dcc8a7b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:02.103517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316207b3-727b-4372-b98f-cf8d6dcc8a7b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:02.103641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316207b3-727b-4372-b98f-cf8d6dcc8a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:02.172438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-316207b3-727b-4372-b98f-cf8d6dcc8a7b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:02.173653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-316207b3-727b-4372-b98f-cf8d6dcc8a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:02.271530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 655/1327] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:55:02.308998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121cb52-fc6e-430b-b2d6-931c6708e805 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:02.310541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0121cb52-fc6e-430b-b2d6-931c6708e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:02.373660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:02.416729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121cb52-fc6e-430b-b2d6-931c6708e805 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:02.489369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:02.489974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:02.490251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:02.490520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:02.492135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:02.606425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:02.609301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:02.611218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9633eb8-e54f-476a-af67-2aff96dffab2 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:02.629633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:04.737260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c0b9b-23f4-4ea7-8056-d9ca69f52a00 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:04.737825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c0b9b-23f4-4ea7-8056-d9ca69f52a00 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:04.737825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c0b9b-23f4-4ea7-8056-d9ca69f52a00 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:04.739164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 656/1329] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:04.744356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac70e-babe-4c0d-8362-e0bfe1ddf09d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:04.744754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac70e-babe-4c0d-8362-e0bfe1ddf09d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:04.744917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac70e-babe-4c0d-8362-e0bfe1ddf09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:04.811733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-664ac70e-babe-4c0d-8362-e0bfe1ddf09d None None] MFA Rules not processed for user `e5ba3c8cd91e4ef6b8737d677d96472d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:04.812593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-664ac70e-babe-4c0d-8362-e0bfe1ddf09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:04.916341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1330] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:04.961073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c0b9b-23f4-4ea7-8056-d9ca69f52a00 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:04.962657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-713c0b9b-23f4-4ea7-8056-d9ca69f52a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:04.995727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:05.069130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c0b9b-23f4-4ea7-8056-d9ca69f52a00 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:05.142299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:05.142908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:05.143186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:05.143458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:05.145087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:05.224306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:05.256687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:05.258118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e762af9-5cef-4277-93cb-4218b45f37c6 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:05.275704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 657/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:07.557112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e622b-b60d-49a5-a5b4-29315d91e351 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:07.557666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e622b-b60d-49a5-a5b4-29315d91e351 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:07.557666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e622b-b60d-49a5-a5b4-29315d91e351 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:07.559006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1332] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:07.564264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fdc4f-9501-40d5-a99e-c1011fc1fc6a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:07.564695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fdc4f-9501-40d5-a99e-c1011fc1fc6a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:07.564814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fdc4f-9501-40d5-a99e-c1011fc1fc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:07.630500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3b3fdc4f-9501-40d5-a99e-c1011fc1fc6a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:07.631722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b3fdc4f-9501-40d5-a99e-c1011fc1fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:07.728940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 658/1333] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:07.765531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e622b-b60d-49a5-a5b4-29315d91e351 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:07.767091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a37e622b-b60d-49a5-a5b4-29315d91e351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:07.835192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:07.870236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e622b-b60d-49a5-a5b4-29315d91e351 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:07.943050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:07.943688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:07.943968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:07.944240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:07.945834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:08.052466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:08.053789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e674514-a7a5-4ef2-b772-b11944a55ed5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:08.064185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:08.072254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:10.206864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0204-2c28-4a54-afde-10add825c5b8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:10.207398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0204-2c28-4a54-afde-10add825c5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:10.207398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0204-2c28-4a54-afde-10add825c5b8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:10.208796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 659/1335] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:10.213968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92face-3e52-40a1-8dfe-2183f01e9eca None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:10.214389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92face-3e52-40a1-8dfe-2183f01e9eca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:10.214544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92face-3e52-40a1-8dfe-2183f01e9eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:10.280522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3c92face-3e52-40a1-8dfe-2183f01e9eca None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:10.281725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c92face-3e52-40a1-8dfe-2183f01e9eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:10.377301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1336] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:10.418233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0204-2c28-4a54-afde-10add825c5b8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:10.420380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-350b0204-2c28-4a54-afde-10add825c5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:10.460635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:10.522994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0204-2c28-4a54-afde-10add825c5b8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:10.593999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:10.594590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:10.594859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:10.595123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:10.596734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:10.680857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:10.706176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:10.707620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482746b-2b93-4673-894f-f29ade86b013 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:10.725605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 660/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:13.057823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d428d98-e7fc-4c3a-9e8d-eedaddd2ead5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:13.058393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d428d98-e7fc-4c3a-9e8d-eedaddd2ead5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:13.058393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d428d98-e7fc-4c3a-9e8d-eedaddd2ead5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:13.059750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1338] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:13.065506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b2bfd-7c47-462c-843a-ac3efc594319 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:13.065928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b2bfd-7c47-462c-843a-ac3efc594319 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:13.066040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b2bfd-7c47-462c-843a-ac3efc594319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:13.131143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ba2b2bfd-7c47-462c-843a-ac3efc594319 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:13.132348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba2b2bfd-7c47-462c-843a-ac3efc594319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:13.226040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 661/1339] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:13.263660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d428d98-e7fc-4c3a-9e8d-eedaddd2ead5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:13.265193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d428d98-e7fc-4c3a-9e8d-eedaddd2ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:13.340067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:13.369624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d428d98-e7fc-4c3a-9e8d-eedaddd2ead5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:13.444317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:13.444931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:13.445217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:13.445491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:13.447138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:13.564790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:13.566244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81536d08-ab95-4dd6-890c-24077d0930c9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:13.584495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:17.548153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec9ae-f493-4da5-9072-f046b11946f6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:17.548713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec9ae-f493-4da5-9072-f046b11946f6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:17.548713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec9ae-f493-4da5-9072-f046b11946f6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:17.550121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 662/1341] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:17.555349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0aa1c-907c-4e15-ba88-172cdd09824a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:17.555808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0aa1c-907c-4e15-ba88-172cdd09824a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:17.555954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0aa1c-907c-4e15-ba88-172cdd09824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:17.561936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:17.631072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-75c0aa1c-907c-4e15-ba88-172cdd09824a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:17.632207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75c0aa1c-907c-4e15-ba88-172cdd09824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:17.726664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1342] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:17.766886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec9ae-f493-4da5-9072-f046b11946f6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:17.768481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-688ec9ae-f493-4da5-9072-f046b11946f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:17.817978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:17.878879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec9ae-f493-4da5-9072-f046b11946f6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:17.952453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:17.953052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:17.953323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:17.953587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:17.955165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:18.049859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:18.066627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:18.068105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514a76d-e1bd-4e93-967f-ed095463e2cd service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:18.086101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 663/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:20.774767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc781d-fcf7-448c-9d7f-bd46cd866c99 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:20.775353 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc781d-fcf7-448c-9d7f-bd46cd866c99 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:20.775353 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc781d-fcf7-448c-9d7f-bd46cd866c99 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:20.776699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1344] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:20.782207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177628-dc0d-4cb7-8b17-042026dd5ed9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:20.782638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177628-dc0d-4cb7-8b17-042026dd5ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:20.782882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177628-dc0d-4cb7-8b17-042026dd5ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:20.849809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ca177628-dc0d-4cb7-8b17-042026dd5ed9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:20.851011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ca177628-dc0d-4cb7-8b17-042026dd5ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:20.951489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 664/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:20.989513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc781d-fcf7-448c-9d7f-bd46cd866c99 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:20.991066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ebdc781d-fcf7-448c-9d7f-bd46cd866c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:21.079328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:21.097444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc781d-fcf7-448c-9d7f-bd46cd866c99 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:21.174338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:21.174949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:21.175230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:21.175502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:21.177150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:21.289615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:21.291759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ac7f1-40f0-4e82-bde6-5ebadc4935f3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:21.313974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:25.318443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045c39-9f53-47e2-9b6c-af2f912eba4c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:25.319000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045c39-9f53-47e2-9b6c-af2f912eba4c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:25.319000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045c39-9f53-47e2-9b6c-af2f912eba4c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:25.320408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 665/1347] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:25.325757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6213c-ca4f-44f9-b04b-de0a146797a4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:25.326176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6213c-ca4f-44f9-b04b-de0a146797a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:25.326309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6213c-ca4f-44f9-b04b-de0a146797a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:25.341377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:25.397231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-73d6213c-ca4f-44f9-b04b-de0a146797a4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:25.398482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73d6213c-ca4f-44f9-b04b-de0a146797a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:25.514041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:25.554743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3045c39-9f53-47e2-9b6c-af2f912eba4c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:25.556361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3045c39-9f53-47e2-9b6c-af2f912eba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:25.618173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:25.669327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3045c39-9f53-47e2-9b6c-af2f912eba4c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:25.744615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:25.745236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:25.745517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:25.745789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:25.748063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:25.866103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:25.870209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:25.871880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc092d-9d5d-484e-8c0f-a36014a64b00 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:25.892084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 666/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:30.811865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574d6e6-4367-4fcb-90d2-0969fe167e5d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:30.813006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574d6e6-4367-4fcb-90d2-0969fe167e5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:30.813006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574d6e6-4367-4fcb-90d2-0969fe167e5d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:30.820416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1350] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 06:55:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:30.837272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236f094-c222-447f-b4b9-da45091c2c66 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:30.837806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236f094-c222-447f-b4b9-da45091c2c66 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:30.838068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236f094-c222-447f-b4b9-da45091c2c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:30.951516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2236f094-c222-447f-b4b9-da45091c2c66 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:30.953277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2236f094-c222-447f-b4b9-da45091c2c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:31.074293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 667/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 06:55:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:35.251018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5966d-e8b3-4e18-a362-28f9501676d3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:35.251569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5966d-e8b3-4e18-a362-28f9501676d3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:35.251569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5966d-e8b3-4e18-a362-28f9501676d3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:35.253107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1352] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 06:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:35.257972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef762c-e8d9-4c50-be5c-98a45be05cfb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:35.258911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef762c-e8d9-4c50-be5c-98a45be05cfb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:35.259469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef762c-e8d9-4c50-be5c-98a45be05cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:35.326421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:35.356331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-caef762c-e8d9-4c50-be5c-98a45be05cfb None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:35.357945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-caef762c-e8d9-4c50-be5c-98a45be05cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:35.488003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 668/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 06:55:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:35.511467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=59 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:36.722371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf13d-bbe1-4272-a16b-2eacfa3b19f9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:36.723013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf13d-bbe1-4272-a16b-2eacfa3b19f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:36.723013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf13d-bbe1-4272-a16b-2eacfa3b19f9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:36.724308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1354] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 06:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:36.729048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa865a-3246-4d9a-8b47-2230ebfabd62 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:36.729478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa865a-3246-4d9a-8b47-2230ebfabd62 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:36.729714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa865a-3246-4d9a-8b47-2230ebfabd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:36.806776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-17fa865a-3246-4d9a-8b47-2230ebfabd62 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:36.808006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17fa865a-3246-4d9a-8b47-2230ebfabd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:36.854560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:36.922107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 669/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 06:55:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:40.321129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3457-496d-4fa8-b1bc-5f60fda7f843 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:40.321645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3457-496d-4fa8-b1bc-5f60fda7f843 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:40.321645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3457-496d-4fa8-b1bc-5f60fda7f843 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:40.326953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1356] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:40.329785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef196-cc8e-491c-b0b2-3c2fdbfef007 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:40.330247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef196-cc8e-491c-b0b2-3c2fdbfef007 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:40.330350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef196-cc8e-491c-b0b2-3c2fdbfef007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:40.407519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-541ef196-cc8e-491c-b0b2-3c2fdbfef007 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:40.408751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-541ef196-cc8e-491c-b0b2-3c2fdbfef007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:40.515725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 670/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:40.564828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b27a40-ea6c-489f-83a2-61db56311f6b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:40.565112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b27a40-ea6c-489f-83a2-61db56311f6b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:40.565386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b27a40-ea6c-489f-83a2-61db56311f6b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:40.566717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1358] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 9 06:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:40.576242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4320a4-fd7a-4cc6-8695-7220dbced634 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:40.576542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4320a4-fd7a-4cc6-8695-7220dbced634 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:40.576809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4320a4-fd7a-4cc6-8695-7220dbced634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:40.598473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:40.665525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dd4320a4-fd7a-4cc6-8695-7220dbced634 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:40.666727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd4320a4-fd7a-4cc6-8695-7220dbced634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:40.790598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 671/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 9 06:55:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:40.798506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b27a40-ea6c-489f-83a2-61db56311f6b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:40.800091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19b27a40-ea6c-489f-83a2-61db56311f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:40.945707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b27a40-ea6c-489f-83a2-61db56311f6b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:41.018694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:41.064416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:41.065026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:41.065366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:41.065705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:41.067346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:41.160632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:41.162227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ec5dc-abad-4d17-8fba-751d98d57936 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:41.181401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:43.414552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6b20e-0dff-43d7-8a5d-eda97c3d6d69 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:43.415023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6b20e-0dff-43d7-8a5d-eda97c3d6d69 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:43.415104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6b20e-0dff-43d7-8a5d-eda97c3d6d69 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:43.416613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 672/1361] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:43.421886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff262-803a-443d-b4da-434d93c03c8c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:43.422490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff262-803a-443d-b4da-434d93c03c8c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:43.422490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff262-803a-443d-b4da-434d93c03c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:43.507901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9fcff262-803a-443d-b4da-434d93c03c8c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:43.509256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fcff262-803a-443d-b4da-434d93c03c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:43.554182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:43.623930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:43.670147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6b20e-0dff-43d7-8a5d-eda97c3d6d69 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:43.671706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7be6b20e-0dff-43d7-8a5d-eda97c3d6d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:43.748851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:43.844371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6b20e-0dff-43d7-8a5d-eda97c3d6d69 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:43.983086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:43.983736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:43.984020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:43.984614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:43.985885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:44.010629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:44.085252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:44.086901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1142a92-d63c-442b-bd78-85b04236671b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:44.107339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 673/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:46.456470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1b0ef-7cb2-471d-9f5a-9f952eaa1730 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:46.456982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1b0ef-7cb2-471d-9f5a-9f952eaa1730 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:46.457062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1b0ef-7cb2-471d-9f5a-9f952eaa1730 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:46.458682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1364] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:46.463512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfcaa6-24f6-4cf8-919f-fb78cdb360c9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:46.463993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfcaa6-24f6-4cf8-919f-fb78cdb360c9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:46.464105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfcaa6-24f6-4cf8-919f-fb78cdb360c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:46.539133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dfdfcaa6-24f6-4cf8-919f-fb78cdb360c9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:46.540370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dfdfcaa6-24f6-4cf8-919f-fb78cdb360c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:46.657539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:46.663769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 674/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:46.709192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab7dfd-7852-4f10-9f7a-43ea373e954e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:46.709480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab7dfd-7852-4f10-9f7a-43ea373e954e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:46.709809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab7dfd-7852-4f10-9f7a-43ea373e954e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:46.711206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1366] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 9 06:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:46.717095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda855a-26b4-461f-9f3d-139d6c99a17a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:46.717388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda855a-26b4-461f-9f3d-139d6c99a17a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:46.717668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda855a-26b4-461f-9f3d-139d6c99a17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:46.810343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-acda855a-26b4-461f-9f3d-139d6c99a17a None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:46.811716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-acda855a-26b4-461f-9f3d-139d6c99a17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:46.946213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 675/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 9 06:55:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:46.954581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab7dfd-7852-4f10-9f7a-43ea373e954e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:46.956153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24ab7dfd-7852-4f10-9f7a-43ea373e954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:47.113539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab7dfd-7852-4f10-9f7a-43ea373e954e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:47.134813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:47.235951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:47.236487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:47.236779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:47.237049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:47.238668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:47.333580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:47.335173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e274e-9f81-402e-80e2-2f42f8304d06 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:47.354366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:48.390263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885a7a2-2d26-43f0-8a35-290f110dc746 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:48.390773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885a7a2-2d26-43f0-8a35-290f110dc746 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:48.390877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885a7a2-2d26-43f0-8a35-290f110dc746 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:48.395633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 676/1369] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:48.397750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e61348-26a8-4ce2-b4ae-58cc81afb857 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:48.398162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e61348-26a8-4ce2-b4ae-58cc81afb857 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:48.398344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e61348-26a8-4ce2-b4ae-58cc81afb857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:48.433764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:48.482864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f7e61348-26a8-4ce2-b4ae-58cc81afb857 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:48.485412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7e61348-26a8-4ce2-b4ae-58cc81afb857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:48.618676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:48.965203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5baafb-169b-45ea-ad1e-1ee564aa8d97 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:48.965487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5baafb-169b-45ea-ad1e-1ee564aa8d97 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:48.965761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5baafb-169b-45ea-ad1e-1ee564aa8d97 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:48.968119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 677/1371] 10.222.0.30 () {54 vars in 964 bytes} [Tue Aug 9 06:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:48.972273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca963d-5544-42a2-859a-e689e3066d4d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:48.972629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca963d-5544-42a2-859a-e689e3066d4d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:48.972972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca963d-5544-42a2-859a-e689e3066d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:49.060290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-57ca963d-5544-42a2-859a-e689e3066d4d None None] MFA Rules not processed for user `6c7d784c96b044ef8b383767e8c1cb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:49.061527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57ca963d-5544-42a2-859a-e689e3066d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:49.113307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:49.146076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Aug 9 06:55:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:55:49.155100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5baafb-169b-45ea-ad1e-1ee564aa8d97 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:49.156744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0b5baafb-169b-45ea-ad1e-1ee564aa8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:49.211475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:49.271204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5baafb-169b-45ea-ad1e-1ee564aa8d97 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:49.348302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:49.348915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:49.349192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:49.349464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:49.351133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:49.453223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:49.468160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:49.469567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:49.469687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258809f-e520-4c67-959a-b17607d9ce40 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:49.470106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258809f-e520-4c67-959a-b17607d9ce40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:49.470195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258809f-e520-4c67-959a-b17607d9ce40 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:49.471529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1373] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:55:49.476978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d5869-1460-4291-b36d-ed81905f79ae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:49.477486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d5869-1460-4291-b36d-ed81905f79ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:49.477952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d5869-1460-4291-b36d-ed81905f79ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:49.488594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 678/1374] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:49.565540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-768d5869-1460-4291-b36d-ed81905f79ae None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:49.566657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-768d5869-1460-4291-b36d-ed81905f79ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:49.680437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:55:49.727916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:49.729051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:49.856564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6d14b-9725-4e4b-8c32-0391f8d777d5 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:49.978227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:49.991192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:49.991834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:49.992115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:49.992386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:49.993994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:50.083847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:50.085434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea5f41-90fb-4eed-8008-981deae79a15 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:50.104271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 679/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:52.169165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d0a20-903b-47ec-a04b-274f82aa747f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:52.169736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d0a20-903b-47ec-a04b-274f82aa747f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:52.169736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d0a20-903b-47ec-a04b-274f82aa747f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:52.171072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1377] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 06:55:52.176400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d5aea-51eb-4ae2-af20-a0c418e65856 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:52.176852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d5aea-51eb-4ae2-af20-a0c418e65856 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:52.176961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d5aea-51eb-4ae2-af20-a0c418e65856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:52.242832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-708d5aea-51eb-4ae2-af20-a0c418e65856 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:52.244076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-708d5aea-51eb-4ae2-af20-a0c418e65856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:52.315904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:52.355835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 680/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:55:52.401355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d0a20-903b-47ec-a04b-274f82aa747f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:52.403831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef3d0a20-903b-47ec-a04b-274f82aa747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:52.468667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:52.573113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d0a20-903b-47ec-a04b-274f82aa747f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:52.692335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:52.692944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:52.693221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:52.693488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:52.695069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:52.707413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:52.794719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:52.796320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6c24a-b0a2-4dd8-b8c4-955fd8f90993 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:52.816444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:54.906004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e8aa8-6768-4a55-8d46-6d2ef119a152 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:54.906562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e8aa8-6768-4a55-8d46-6d2ef119a152 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:54.906562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e8aa8-6768-4a55-8d46-6d2ef119a152 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:54.908068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 681/1380] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:54.913155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4d0d-431e-4673-b33b-a89b1fa3a028 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:54.913570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4d0d-431e-4673-b33b-a89b1fa3a028 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:54.913795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4d0d-431e-4673-b33b-a89b1fa3a028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:54.988615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-673d4d0d-431e-4673-b33b-a89b1fa3a028 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:54.989891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-673d4d0d-431e-4673-b33b-a89b1fa3a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:55.087362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:55.101561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:55.147508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e8aa8-6768-4a55-8d46-6d2ef119a152 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:55.149492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d94e8aa8-6768-4a55-8d46-6d2ef119a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:55.308664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e8aa8-6768-4a55-8d46-6d2ef119a152 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:55.360222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:55.429875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:55.430577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:55.430924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:55.431262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:55.433602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:55.528239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:55.529850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7919bb-8c16-4371-8df5-b00f1c40ed46 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:55.550372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 682/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:55:57.863382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90dcc8-68b5-4902-b492-26a9ff819750 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:57.863884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90dcc8-68b5-4902-b492-26a9ff819750 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:57.863998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90dcc8-68b5-4902-b492-26a9ff819750 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:57.865341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1383] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:55:57.872889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea813e-702f-40f0-8b6f-83f37889ac36 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:57.873582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea813e-702f-40f0-8b6f-83f37889ac36 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:57.873977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea813e-702f-40f0-8b6f-83f37889ac36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:57.964133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dfea813e-702f-40f0-8b6f-83f37889ac36 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:55:57.965819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dfea813e-702f-40f0-8b6f-83f37889ac36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:57.968688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:58.091422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 683/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:55:58.149194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90dcc8-68b5-4902-b492-26a9ff819750 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:55:58.150737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa90dcc8-68b5-4902-b492-26a9ff819750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:58.313081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90dcc8-68b5-4902-b492-26a9ff819750 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:55:58.384217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:55:58.437315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:55:58.437932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:55:58.438214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:55:58.438476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:55:58.440136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:55:58.538567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:55:58.540178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df9299-76ef-46ca-b99c-7de3456d481f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:55:58.560315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:00.636956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5efdc9-ec69-47f8-997e-2d4f0adef4ea None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:00.637462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5efdc9-ec69-47f8-997e-2d4f0adef4ea None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:00.637462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5efdc9-ec69-47f8-997e-2d4f0adef4ea None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:00.640084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 684/1386] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:00.645668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deeb7e-f315-424e-a41e-dd839a4f593d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:00.646291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deeb7e-f315-424e-a41e-dd839a4f593d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:00.646291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deeb7e-f315-424e-a41e-dd839a4f593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:00.716431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-12deeb7e-f315-424e-a41e-dd839a4f593d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:00.718026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12deeb7e-f315-424e-a41e-dd839a4f593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:00.752076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:00.833866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:00.878916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5efdc9-ec69-47f8-997e-2d4f0adef4ea None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:00.880604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd5efdc9-ec69-47f8-997e-2d4f0adef4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:01.016021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:01.038447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5efdc9-ec69-47f8-997e-2d4f0adef4ea None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:01.154636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:01.155244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:01.155528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:01.155831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:01.157458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:01.247409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:01.249198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b962c-0b2d-4ceb-a83c-f7272dea2e9b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:01.254086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:01.268034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 685/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:03.317939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6125-af4d-4980-bbac-00a2baef1eb9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:03.318502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6125-af4d-4980-bbac-00a2baef1eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:03.318502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6125-af4d-4980-bbac-00a2baef1eb9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:03.319866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1389] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:03.325978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832db7f-c314-4814-9a5b-c290e6d68d39 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:03.326421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832db7f-c314-4814-9a5b-c290e6d68d39 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:03.326580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832db7f-c314-4814-9a5b-c290e6d68d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:03.393918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4832db7f-c314-4814-9a5b-c290e6d68d39 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:03.395192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4832db7f-c314-4814-9a5b-c290e6d68d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:03.492411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 686/1390] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:03.537200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee6125-af4d-4980-bbac-00a2baef1eb9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:03.538802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58ee6125-af4d-4980-bbac-00a2baef1eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:03.687355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee6125-af4d-4980-bbac-00a2baef1eb9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:03.696444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:03.802766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:03.803368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:03.803675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:03.803950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:03.805550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:03.900406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:03.902597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ceeee-c388-45c0-86b9-38abef5e2c0b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:03.921254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1391] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:04.515831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3e094-6572-4125-8578-1bb7afafeb88 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:04.516373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3e094-6572-4125-8578-1bb7afafeb88 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:04.516373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3e094-6572-4125-8578-1bb7afafeb88 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:04.517726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 687/1392] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:04.522938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db907d1-ec0c-4b23-b9be-52fc57b6b131 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:04.523364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db907d1-ec0c-4b23-b9be-52fc57b6b131 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:04.523492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db907d1-ec0c-4b23-b9be-52fc57b6b131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:04.539639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:04.592401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3db907d1-ec0c-4b23-b9be-52fc57b6b131 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:04.593648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3db907d1-ec0c-4b23-b9be-52fc57b6b131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:04.691685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1393] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:04.748377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcfb08-d758-429c-9063-0758daeb48da None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:04.748734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcfb08-d758-429c-9063-0758daeb48da None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:04.749079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcfb08-d758-429c-9063-0758daeb48da None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:04.750517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 688/1394] 10.222.0.30 () {54 vars in 967 bytes} [Tue Aug 9 06:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:04.755650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e4963-a461-4b46-b3cc-1d0529be6e42 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:04.755943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e4963-a461-4b46-b3cc-1d0529be6e42 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:04.756220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e4963-a461-4b46-b3cc-1d0529be6e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:04.832819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b22e4963-a461-4b46-b3cc-1d0529be6e42 None None] MFA Rules not processed for user `337f5152bd51454cb58d6b2092d5cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:04.834016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b22e4963-a461-4b46-b3cc-1d0529be6e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:04.876586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:04.918829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1395] 10.222.0.30 () {58 vars in 1064 bytes} [Tue Aug 9 06:56:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:56:04.926411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcfb08-d758-429c-9063-0758daeb48da None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:04.927995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-14bcfb08-d758-429c-9063-0758daeb48da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:05.003194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:05.033976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcfb08-d758-429c-9063-0758daeb48da None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:05.111233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:05.111877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:05.112156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:05.112429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:05.114043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:05.225615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:05.227048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf10a6-0d18-4060-a34f-28af54d291df service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:05.242812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:05.246841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 689/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:05.968486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85dab7-cd2b-4f19-bbbf-f3176753a213 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:05.968918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85dab7-cd2b-4f19-bbbf-f3176753a213 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:05.968918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85dab7-cd2b-4f19-bbbf-f3176753a213 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:05.970233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1397] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:05.975583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:05.975988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:05.976314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:05.982652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c87a-30b1-4cf9-9851-ef8cf29642ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:05.983065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c87a-30b1-4cf9-9851-ef8cf29642ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:05.983411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c87a-30b1-4cf9-9851-ef8cf29642ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:06.045377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:06.046598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:06.052840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-45f5c87a-30b1-4cf9-9851-ef8cf29642ee None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:06.054006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45f5c87a-30b1-4cf9-9851-ef8cf29642ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:06.141001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1398] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 06:56:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:56:06.145170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 690/1399] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:06.188438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d904758-f9a8-4a9c-aeb0-aa57820e65e4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:06.188729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d904758-f9a8-4a9c-aeb0-aa57820e65e4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:06.189008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d904758-f9a8-4a9c-aeb0-aa57820e65e4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:06.190586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1400] 10.222.0.30 () {54 vars in 943 bytes} [Tue Aug 9 06:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:06.192143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:06.193320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:06.196290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:06.196574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:06.196854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:06.255070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:06.269550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:06.274658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] MFA Rules not processed for user `8f3a4e99f3044d35aa2e310478f3653b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:06.276443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:06.308873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bb076-bab2-4e3b-9997-7448fa8125a7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:06.387851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1401] 10.222.0.30 () {58 vars in 1040 bytes} [Tue Aug 9 06:56:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:56:06.397223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:06.407854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0e3df-be48-479a-81e3-94401bfe21ea None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:06.428592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:06.429202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:06.429487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:06.429757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:06.431356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:06.506637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:06.509101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:06.514251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:06.514866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:06.515170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:06.515441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:06.517126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:06.527553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:06.529172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc15a8-8fe7-4bed-9a4a-0ec601d5cc4c service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:06.547411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 691/1402] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:06.611337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:06.613211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dafc1-5a81-4dfb-80ae-527be0da33a9 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:06.621997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1403] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 06:56:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 06:56:08.675113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66cbe8-ceac-44fa-9ddb-d8205f469320 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:08.675673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66cbe8-ceac-44fa-9ddb-d8205f469320 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:08.675791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66cbe8-ceac-44fa-9ddb-d8205f469320 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:08.681153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 692/1404] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:08.683948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653de4-d27c-4e8e-a3dc-9a752cb051fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:08.684855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653de4-d27c-4e8e-a3dc-9a752cb051fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:08.685463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653de4-d27c-4e8e-a3dc-9a752cb051fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:08.769012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ae653de4-d27c-4e8e-a3dc-9a752cb051fd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:08.770755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae653de4-d27c-4e8e-a3dc-9a752cb051fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:08.904597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:08.923304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:08.970625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66cbe8-ceac-44fa-9ddb-d8205f469320 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:08.972387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed66cbe8-ceac-44fa-9ddb-d8205f469320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:09.161950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66cbe8-ceac-44fa-9ddb-d8205f469320 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:09.254711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:09.331108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:09.331839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:09.332126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:09.332395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:09.334082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:09.453785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:09.460127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee17647-033b-42ed-ba3d-a61020a4be1c service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:09.488027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 693/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:11.640731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50b59-4826-40d0-9c8f-570b57fc96e8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:11.641340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50b59-4826-40d0-9c8f-570b57fc96e8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:11.641340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50b59-4826-40d0-9c8f-570b57fc96e8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:11.642830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1407] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:11.647744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460319-02a1-4b7c-a712-88be30e6af98 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:11.648169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460319-02a1-4b7c-a712-88be30e6af98 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:11.648296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460319-02a1-4b7c-a712-88be30e6af98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:11.721754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d5460319-02a1-4b7c-a712-88be30e6af98 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:11.722971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5460319-02a1-4b7c-a712-88be30e6af98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:11.732403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:11.832450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 694/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:11.878580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50b59-4826-40d0-9c8f-570b57fc96e8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:11.880135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fea50b59-4826-40d0-9c8f-570b57fc96e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:12.047048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50b59-4826-40d0-9c8f-570b57fc96e8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:12.120057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:12.178153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:12.178766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:12.179046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:12.179317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:12.180975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:12.278554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:12.280599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365396b-d2bb-4091-9d61-e235dd2c5553 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:12.305468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:21.830474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=39 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:22.302865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=67 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:28.738959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd114b-9ed1-4dcf-b9b2-2815e8250fda None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:28.739527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd114b-9ed1-4dcf-b9b2-2815e8250fda None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:28.739527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd114b-9ed1-4dcf-b9b2-2815e8250fda None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:28.740825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 695/1410] 10.222.0.30 () {54 vars in 918 bytes} [Tue Aug 9 06:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:28.746578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b293-2d02-47dc-bd04-c781e9f55294 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:28.747000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b293-2d02-47dc-bd04-c781e9f55294 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:28.747162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b293-2d02-47dc-bd04-c781e9f55294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:28.821262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6d39b293-2d02-47dc-bd04-c781e9f55294 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:28.823280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6d39b293-2d02-47dc-bd04-c781e9f55294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:28.928319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Aug 9 06:56:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:28.943614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09712-b342-45e4-b29c-5a856a10c849 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:28.943919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09712-b342-45e4-b29c-5a856a10c849 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:28.944201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09712-b342-45e4-b29c-5a856a10c849 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:28.945528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 696/1412] 10.222.0.30 () {54 vars in 918 bytes} [Tue Aug 9 06:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:28.950023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27634d5-732c-42eb-a667-9ef6a35d72e2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:28.950310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27634d5-732c-42eb-a667-9ef6a35d72e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:28.950584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27634d5-732c-42eb-a667-9ef6a35d72e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:29.021115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:29.023860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c27634d5-732c-42eb-a667-9ef6a35d72e2 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:29.025799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c27634d5-732c-42eb-a667-9ef6a35d72e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:29.142812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Aug 9 06:56:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:31.615466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e44b0-4e47-4c3d-b7fc-c64f5e87ef5e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:31.616033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e44b0-4e47-4c3d-b7fc-c64f5e87ef5e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:31.616033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e44b0-4e47-4c3d-b7fc-c64f5e87ef5e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:31.617574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:31.623502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e8bf-eb5f-4d92-8536-45246bc500fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:31.623969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e8bf-eb5f-4d92-8536-45246bc500fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:31.624090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e8bf-eb5f-4d92-8536-45246bc500fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:31.690228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2b57e8bf-eb5f-4d92-8536-45246bc500fa None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:31.691437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b57e8bf-eb5f-4d92-8536-45246bc500fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:31.745824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:31.787681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:31.831217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e44b0-4e47-4c3d-b7fc-c64f5e87ef5e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:31.832857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-209e44b0-4e47-4c3d-b7fc-c64f5e87ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:31.984977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e44b0-4e47-4c3d-b7fc-c64f5e87ef5e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:32.058875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:32.098688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:32.099292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:32.099567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:32.099870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:32.101462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:32.192940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:32.194554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79cd9c-6c81-4f3f-97de-0e96bf0d8d17 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:32.212879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 698/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:33.111246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3038-0508-46ac-a279-e426eadf2809 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:33.111800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3038-0508-46ac-a279-e426eadf2809 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:33.111800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3038-0508-46ac-a279-e426eadf2809 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:33.113109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:33.117935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2017f-2026-4952-a7e7-e6860b47eda7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:33.118346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2017f-2026-4952-a7e7-e6860b47eda7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:33.118499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2017f-2026-4952-a7e7-e6860b47eda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:33.194189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-eac2017f-2026-4952-a7e7-e6860b47eda7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:33.195403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eac2017f-2026-4952-a7e7-e6860b47eda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:33.222904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:33.293310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:33.445341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc3038-0508-46ac-a279-e426eadf2809 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:33.446978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8bbc3038-0508-46ac-a279-e426eadf2809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:33.590809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc3038-0508-46ac-a279-e426eadf2809 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:33.630378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:33.702804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:33.703325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:33.703618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:33.703896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:33.705480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:33.794698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:33.796347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984f367-df60-41d8-bf06-69be81747635 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:33.814564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:34.691224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fdc66-5942-455d-804e-efab753b7e0a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:34.691787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fdc66-5942-455d-804e-efab753b7e0a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:34.691787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fdc66-5942-455d-804e-efab753b7e0a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:34.693053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:34.698631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f25bfa-814a-492b-87a2-4fb652d31d7a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:34.699033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f25bfa-814a-492b-87a2-4fb652d31d7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:34.699190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f25bfa-814a-492b-87a2-4fb652d31d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:34.752875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:34.776562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2f25bfa-814a-492b-87a2-4fb652d31d7a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:34.777789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2f25bfa-814a-492b-87a2-4fb652d31d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:34.996111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:35.151930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fdc66-5942-455d-804e-efab753b7e0a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:35.153496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-795fdc66-5942-455d-804e-efab753b7e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:35.301023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fdc66-5942-455d-804e-efab753b7e0a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:35.314952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:35.414370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:35.414985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:35.415294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:35.416233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:35.417841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:35.504945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:35.506526 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ea4e3-fc38-45f4-9516-3dd5a033a86d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:35.524624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 701/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:36.378501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cba8f-935b-4525-87ed-724f5a108d94 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:36.379068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cba8f-935b-4525-87ed-724f5a108d94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:36.379068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cba8f-935b-4525-87ed-724f5a108d94 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:36.380424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:36.385948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f8c24-0b47-4d16-bb79-1270141f9443 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:36.386330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f8c24-0b47-4d16-bb79-1270141f9443 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:36.386495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f8c24-0b47-4d16-bb79-1270141f9443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:36.406759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:36.462424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-069f8c24-0b47-4d16-bb79-1270141f9443 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:36.464419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-069f8c24-0b47-4d16-bb79-1270141f9443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:36.560285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:36.713646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cba8f-935b-4525-87ed-724f5a108d94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:36.715168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa0cba8f-935b-4525-87ed-724f5a108d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:36.821726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:36.858478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cba8f-935b-4525-87ed-724f5a108d94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:36.968688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:36.969217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:36.969493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:36.969761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:36.971362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:37.045616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:37.059822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:37.061407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8135c34-eae0-4ffb-8e09-9beb76f6ecf6 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:37.080127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:37.934675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65478-df22-49a3-aaf0-b6eeb40e7456 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:37.935222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65478-df22-49a3-aaf0-b6eeb40e7456 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:37.935222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65478-df22-49a3-aaf0-b6eeb40e7456 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:37.936589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:37.941439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45cee0-c319-4358-933a-38a3421e5033 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:37.941873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45cee0-c319-4358-933a-38a3421e5033 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:37.941984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45cee0-c319-4358-933a-38a3421e5033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:38.017502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c45cee0-c319-4358-933a-38a3421e5033 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:38.018702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c45cee0-c319-4358-933a-38a3421e5033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:38.114760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:56:38.271912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65478-df22-49a3-aaf0-b6eeb40e7456 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:38.273462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e1a65478-df22-49a3-aaf0-b6eeb40e7456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:38.357463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:38.421772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65478-df22-49a3-aaf0-b6eeb40e7456 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:38.532619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:38.533206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:38.533481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:38.533751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:38.535331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:38.583606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:38.625447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:38.627056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe1dc4-41ff-4506-a6b9-f5c55ee02163 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:38.644918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 704/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:39.455697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa135de-9da2-482b-8580-c280deebba18 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:39.456262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa135de-9da2-482b-8580-c280deebba18 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:39.456262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa135de-9da2-482b-8580-c280deebba18 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:39.457605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:39.462372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8b36-8db6-42d6-8ccd-4a9a7f4f42cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:39.463536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8b36-8db6-42d6-8ccd-4a9a7f4f42cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:39.463536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8b36-8db6-42d6-8ccd-4a9a7f4f42cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:39.538200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3f0e8b36-8db6-42d6-8ccd-4a9a7f4f42cb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:39.539390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3f0e8b36-8db6-42d6-8ccd-4a9a7f4f42cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:39.637516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:39.788309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa135de-9da2-482b-8580-c280deebba18 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:39.789732 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ffa135de-9da2-482b-8580-c280deebba18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:39.838375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:39.933872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa135de-9da2-482b-8580-c280deebba18 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:40.042773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:40.043329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:40.043623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:40.043906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:40.045508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:40.052340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:40.134600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:40.136414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65deb12-af18-4534-b1f0-75d0ea9ed5b0 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:40.154386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:40.953107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3477044-7a40-460a-8f09-c6c6059075a3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:40.953667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3477044-7a40-460a-8f09-c6c6059075a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:40.953667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3477044-7a40-460a-8f09-c6c6059075a3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:40.954976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:40.959850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d7ae-9b04-43c3-bf19-911bfcb804d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:40.960323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d7ae-9b04-43c3-bf19-911bfcb804d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:40.960432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d7ae-9b04-43c3-bf19-911bfcb804d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:41.038492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5593d7ae-9b04-43c3-bf19-911bfcb804d6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:41.039728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5593d7ae-9b04-43c3-bf19-911bfcb804d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:41.108339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:41.137931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:41.292375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3477044-7a40-460a-8f09-c6c6059075a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:41.293967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e3477044-7a40-460a-8f09-c6c6059075a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:41.311335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:41.438850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3477044-7a40-460a-8f09-c6c6059075a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:41.524435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:41.551292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:41.552038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:41.552376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:41.552716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:41.554347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:41.643030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:41.644676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b3d71-5adf-4a76-808a-e98b4ac438eb service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:41.663203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 707/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:42.470792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dc2f0-2c02-4403-94fb-bde2f1fce923 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:42.471321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dc2f0-2c02-4403-94fb-bde2f1fce923 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:42.471321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dc2f0-2c02-4403-94fb-bde2f1fce923 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:42.472717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:42.477582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cba0a-3520-4ab3-872d-7799969fbb3f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:42.477990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cba0a-3520-4ab3-872d-7799969fbb3f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:42.478338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cba0a-3520-4ab3-872d-7799969fbb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:42.554907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6f0cba0a-3520-4ab3-872d-7799969fbb3f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:42.556077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6f0cba0a-3520-4ab3-872d-7799969fbb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:42.597471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:42.653382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:42.805542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dc2f0-2c02-4403-94fb-bde2f1fce923 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:42.807553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-669dc2f0-2c02-4403-94fb-bde2f1fce923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:42.950831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dc2f0-2c02-4403-94fb-bde2f1fce923 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:43.002989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:43.061301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:43.061833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:43.062110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:43.062379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:43.064009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:43.153131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:43.154735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dfae7-9057-415a-849e-6a760b183162 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:43.172796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:43.977920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae2ea7-850b-4e9c-a4e5-dff3d0ea5b73 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:43.978974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae2ea7-850b-4e9c-a4e5-dff3d0ea5b73 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:43.979151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae2ea7-850b-4e9c-a4e5-dff3d0ea5b73 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:43.980541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 06:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:43.985516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28299d68-24c3-4110-a5ec-89579150c097 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:43.985950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28299d68-24c3-4110-a5ec-89579150c097 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:43.986057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28299d68-24c3-4110-a5ec-89579150c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:44.061690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-28299d68-24c3-4110-a5ec-89579150c097 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:44.062891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-28299d68-24c3-4110-a5ec-89579150c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:44.064908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:44.163363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 06:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:44.314550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae2ea7-850b-4e9c-a4e5-dff3d0ea5b73 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:44.316120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-10ae2ea7-850b-4e9c-a4e5-dff3d0ea5b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:44.460291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae2ea7-850b-4e9c-a4e5-dff3d0ea5b73 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:44.487307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:44.571689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:44.573003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:44.573283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:44.573557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:44.575154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:44.664115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:44.665879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da1e3-0021-4bae-84ef-1a8fde249b6f service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:44.683767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 710/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:46.588989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc4f7-89ba-413a-83fe-a0859cd70a80 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:46.589533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc4f7-89ba-413a-83fe-a0859cd70a80 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:46.589533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc4f7-89ba-413a-83fe-a0859cd70a80 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:46.590880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:46.596322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aa4cf-ce7b-4949-b6ff-77507a397726 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:46.596322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aa4cf-ce7b-4949-b6ff-77507a397726 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:46.596767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aa4cf-ce7b-4949-b6ff-77507a397726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:46.640417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:46.667248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a79aa4cf-ce7b-4949-b6ff-77507a397726 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:46.668430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a79aa4cf-ce7b-4949-b6ff-77507a397726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:46.767402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:46.805259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc4f7-89ba-413a-83fe-a0859cd70a80 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:46.807412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42adc4f7-89ba-413a-83fe-a0859cd70a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:46.910847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc4f7-89ba-413a-83fe-a0859cd70a80 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:46.943004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:46.984213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:46.984817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:46.985094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:46.985364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:46.986961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:47.093270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:47.094695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd20ab9-21f7-4791-af93-3c90b8d0eea9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:47.113358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:49.313043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fe73f-8897-4be8-b117-d73ae1646e53 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:49.313588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fe73f-8897-4be8-b117-d73ae1646e53 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:49.313588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fe73f-8897-4be8-b117-d73ae1646e53 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:49.314927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:49.320276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777297-a3af-459d-89d3-af1749aeb756 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:49.320706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777297-a3af-459d-89d3-af1749aeb756 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:49.320861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777297-a3af-459d-89d3-af1749aeb756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:49.387387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-13777297-a3af-459d-89d3-af1749aeb756 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:49.388626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13777297-a3af-459d-89d3-af1749aeb756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:49.401614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:49.485030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:49.523830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fe73f-8897-4be8-b117-d73ae1646e53 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:49.525537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b09fe73f-8897-4be8-b117-d73ae1646e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:49.628576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fe73f-8897-4be8-b117-d73ae1646e53 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:49.640958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:49.701745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:49.702350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:49.702626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:49.702893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:49.704499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:49.814042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:49.815732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf04fd-2a39-441d-b89c-9844116f000b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:49.833974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 713/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:52.160821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5efe-ce68-4b9e-8a7d-f08660f0b7c3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:52.161375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5efe-ce68-4b9e-8a7d-f08660f0b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:52.161375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5efe-ce68-4b9e-8a7d-f08660f0b7c3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:52.162693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:52.168822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8654b-e37f-4a4b-85ad-eb5671e75a83 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:52.169325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8654b-e37f-4a4b-85ad-eb5671e75a83 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:52.169423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8654b-e37f-4a4b-85ad-eb5671e75a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:52.217504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:52.235761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b3e8654b-e37f-4a4b-85ad-eb5671e75a83 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:52.236983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b3e8654b-e37f-4a4b-85ad-eb5671e75a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:52.334844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:56:52.374874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5efe-ce68-4b9e-8a7d-f08660f0b7c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:52.376435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d5ad5efe-ce68-4b9e-8a7d-f08660f0b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:52.480437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5efe-ce68-4b9e-8a7d-f08660f0b7c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:52.518721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:52.552919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:52.553697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:52.553975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:52.554249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:52.555869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:52.661469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:52.662899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eafb2e-f8ea-487d-90a0-86807af8b88e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:52.681106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:54.793850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba9980-7106-4ac2-951a-f4f1ea4b8f91 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:54.794358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba9980-7106-4ac2-951a-f4f1ea4b8f91 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:54.794358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba9980-7106-4ac2-951a-f4f1ea4b8f91 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:54.795714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:54.800954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f502-693d-47d6-8223-a8216e3642a9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:54.801377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f502-693d-47d6-8223-a8216e3642a9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:54.801511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f502-693d-47d6-8223-a8216e3642a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:54.868298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3299f502-693d-47d6-8223-a8216e3642a9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:54.869803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3299f502-693d-47d6-8223-a8216e3642a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:54.890325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:54.966891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:55.006157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba9980-7106-4ac2-951a-f4f1ea4b8f91 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:55.007772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ecba9980-7106-4ac2-951a-f4f1ea4b8f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:55.111471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba9980-7106-4ac2-951a-f4f1ea4b8f91 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:55.131508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:55.185475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:55.186076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:55.186355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:55.186624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:55.188256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:55.296557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:55.298006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9124a3-7780-44a7-8392-5e8086988777 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:55.315792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 716/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:57.362962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c98bb-43bd-40fd-8349-a0cdc6d3c761 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:57.363524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c98bb-43bd-40fd-8349-a0cdc6d3c761 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:57.363524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c98bb-43bd-40fd-8349-a0cdc6d3c761 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:57.364871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:57.370796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0033a8-5b8f-4666-b8ba-5928c25bb488 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:57.371183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0033a8-5b8f-4666-b8ba-5928c25bb488 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:57.371346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0033a8-5b8f-4666-b8ba-5928c25bb488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:57.434307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:57.437078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5b0033a8-5b8f-4666-b8ba-5928c25bb488 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:56:57.438955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5b0033a8-5b8f-4666-b8ba-5928c25bb488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:57.534760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:56:57.573998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c98bb-43bd-40fd-8349-a0cdc6d3c761 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:56:57.575564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eb3c98bb-43bd-40fd-8349-a0cdc6d3c761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:57.677540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c98bb-43bd-40fd-8349-a0cdc6d3c761 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:56:57.725209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:56:57.750393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:56:57.750994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:57.751279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:57.751577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:57.753229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:56:57.860059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:56:57.860990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd92b0-0465-49be-8cae-89a45ef02567 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:56:57.878989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:56:59.934843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4a41-3f66-4e3d-b660-4112cbfb1f32 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:59.935398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4a41-3f66-4e3d-b660-4112cbfb1f32 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:59.935507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4a41-3f66-4e3d-b660-4112cbfb1f32 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:56:59.936839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:56:59.941975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523845a-dd48-4a0e-a3e6-6ccf0338ff9c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:56:59.942352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523845a-dd48-4a0e-a3e6-6ccf0338ff9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:56:59.942534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523845a-dd48-4a0e-a3e6-6ccf0338ff9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:00.010737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5523845a-dd48-4a0e-a3e6-6ccf0338ff9c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:00.011987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5523845a-dd48-4a0e-a3e6-6ccf0338ff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:00.041442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:00.109601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:00.147731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4a41-3f66-4e3d-b660-4112cbfb1f32 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:00.149256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f04f4a41-3f66-4e3d-b660-4112cbfb1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:00.255147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4a41-3f66-4e3d-b660-4112cbfb1f32 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:00.283154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:00.329579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:00.330187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:00.330469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:00.330745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:00.332393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:00.441228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:00.442664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb0533-3fdc-42f5-8092-ded8c56b4b75 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:00.460709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 719/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:02.522737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4b8e-0d5e-4eba-ba71-5c6c7dd30298 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:02.523271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4b8e-0d5e-4eba-ba71-5c6c7dd30298 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:02.523387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4b8e-0d5e-4eba-ba71-5c6c7dd30298 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:02.524720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:02.530599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b048-8ac3-4eab-8408-91aeef47963d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:02.531049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b048-8ac3-4eab-8408-91aeef47963d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:02.531172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b048-8ac3-4eab-8408-91aeef47963d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:02.597726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0243b048-8ac3-4eab-8408-91aeef47963d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:02.598871 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0243b048-8ac3-4eab-8408-91aeef47963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:02.607223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:02.696239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:02.735751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4b8e-0d5e-4eba-ba71-5c6c7dd30298 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:02.737294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08ac4b8e-0d5e-4eba-ba71-5c6c7dd30298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:02.841143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4b8e-0d5e-4eba-ba71-5c6c7dd30298 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:02.897459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:02.916440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:02.917041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:02.917325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:02.917595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:02.919198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:03.025937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:03.027358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1024e-52cc-4f87-9645-6e3e79a0d6a1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:03.045250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:05.202323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb007c-b8b3-4c09-a2cb-983fcc43ef0f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:05.202852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb007c-b8b3-4c09-a2cb-983fcc43ef0f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:05.202852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb007c-b8b3-4c09-a2cb-983fcc43ef0f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:05.204258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:05.209311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f6d6-12d2-475f-b0f2-ce99eed5ecab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:05.209714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f6d6-12d2-475f-b0f2-ce99eed5ecab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:05.209859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f6d6-12d2-475f-b0f2-ce99eed5ecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:05.276431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c09f6d6-12d2-475f-b0f2-ce99eed5ecab None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:05.277637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c09f6d6-12d2-475f-b0f2-ce99eed5ecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:05.315495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:05.374869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:05.412540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb007c-b8b3-4c09-a2cb-983fcc43ef0f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:05.414093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-50cb007c-b8b3-4c09-a2cb-983fcc43ef0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:05.516883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb007c-b8b3-4c09-a2cb-983fcc43ef0f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:05.555186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:05.591756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:05.592356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:05.592632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:05.592903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:05.594511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:05.701804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:05.703236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8a71d-87e4-4cb7-994b-e59bc411da3b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:05.721752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 722/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:07.743718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe98ce-6baa-48f9-abb0-9b3450850177 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:07.744539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe98ce-6baa-48f9-abb0-9b3450850177 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:07.744539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe98ce-6baa-48f9-abb0-9b3450850177 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:07.745802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:07.751393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028ff6e-b653-469a-b6d8-a9a6974a78aa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:07.751849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028ff6e-b653-469a-b6d8-a9a6974a78aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:07.751977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028ff6e-b653-469a-b6d8-a9a6974a78aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:07.818386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9028ff6e-b653-469a-b6d8-a9a6974a78aa None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:07.819588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9028ff6e-b653-469a-b6d8-a9a6974a78aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:07.840062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:07.918022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:07.954387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe98ce-6baa-48f9-abb0-9b3450850177 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:07.955992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61fe98ce-6baa-48f9-abb0-9b3450850177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:08.060516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe98ce-6baa-48f9-abb0-9b3450850177 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:08.126453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:08.133560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:08.134171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:08.134513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:08.134887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:08.136538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:08.244971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:08.246408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bccad3-1a0a-4c8c-85bb-ce3ad0b8e593 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:08.264455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:10.273613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4d9b-6236-482c-b3fd-3292f812d0af None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:10.274169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4d9b-6236-482c-b3fd-3292f812d0af None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:10.274169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4d9b-6236-482c-b3fd-3292f812d0af None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:10.276128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 06:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:10.281308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215af411-a793-4597-91a0-57f5783b97c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:10.281706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215af411-a793-4597-91a0-57f5783b97c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:10.281869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215af411-a793-4597-91a0-57f5783b97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:10.346953 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-215af411-a793-4597-91a0-57f5783b97c0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:10.348185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-215af411-a793-4597-91a0-57f5783b97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:10.395986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:10.444808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 06:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:10.482286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb4d9b-6236-482c-b3fd-3292f812d0af None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:10.483908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0dbb4d9b-6236-482c-b3fd-3292f812d0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:10.587045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb4d9b-6236-482c-b3fd-3292f812d0af None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:10.633741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:10.660627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:10.661233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:10.661513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:10.661788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:10.663421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:10.771269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:10.772769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05667336-61e9-42c8-8a42-6ce2ea3e42cb service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:10.790431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 725/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:20.443008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=22 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:20.788239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=66 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:30.512676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9583-b428-4a3b-a693-802904a37354 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:30.513233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9583-b428-4a3b-a693-802904a37354 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:30.513233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9583-b428-4a3b-a693-802904a37354 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:30.514658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1471] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:30.519951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d4385-c3ab-4ae9-9ad7-71fbd5522ead None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:30.520371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d4385-c3ab-4ae9-9ad7-71fbd5522ead None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:30.520506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d4385-c3ab-4ae9-9ad7-71fbd5522ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:30.585623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-729d4385-c3ab-4ae9-9ad7-71fbd5522ead None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:30.586831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-729d4385-c3ab-4ae9-9ad7-71fbd5522ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:30.684087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 726/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:30.768075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be1173-b3e7-40e1-a8cf-9e50c9c2647d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:30.768394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be1173-b3e7-40e1-a8cf-9e50c9c2647d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:30.768672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be1173-b3e7-40e1-a8cf-9e50c9c2647d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:30.769998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1473] 10.222.0.30 () {54 vars in 967 bytes} [Tue Aug 9 06:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:30.774776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0a283-20f2-483d-882f-e76b0a014c6c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:30.775189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0a283-20f2-483d-882f-e76b0a014c6c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:30.775467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0a283-20f2-483d-882f-e76b0a014c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:30.850000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7ef0a283-20f2-483d-882f-e76b0a014c6c None None] MFA Rules not processed for user `337f5152bd51454cb58d6b2092d5cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:30.851193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7ef0a283-20f2-483d-882f-e76b0a014c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:30.872437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:30.927931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 727/1474] 10.222.0.30 () {58 vars in 1064 bytes} [Tue Aug 9 06:57:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:30.936080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be1173-b3e7-40e1-a8cf-9e50c9c2647d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:30.937561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-36be1173-b3e7-40e1-a8cf-9e50c9c2647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:31.042336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be1173-b3e7-40e1-a8cf-9e50c9c2647d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:31.114734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:31.115367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:31.115679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:31.115956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:31.117575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:31.171351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:31.226095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:31.227444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd321a-17c9-4d57-928b-b5e35da6cc88 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:31.246440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:33.317512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd93e4-f3af-4645-a295-a2bcfbaa6bc7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:33.318085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd93e4-f3af-4645-a295-a2bcfbaa6bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:33.318085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd93e4-f3af-4645-a295-a2bcfbaa6bc7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:33.319430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 728/1476] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:33.324624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19fdf-277f-451f-8e28-e677c7ee3f05 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:33.325025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19fdf-277f-451f-8e28-e677c7ee3f05 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:33.325163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19fdf-277f-451f-8e28-e677c7ee3f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:33.389527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-24c19fdf-277f-451f-8e28-e677c7ee3f05 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:33.390723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24c19fdf-277f-451f-8e28-e677c7ee3f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:33.485124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:33.508689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd93e4-f3af-4645-a295-a2bcfbaa6bc7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:33.510730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31dd93e4-f3af-4645-a295-a2bcfbaa6bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:33.635201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd93e4-f3af-4645-a295-a2bcfbaa6bc7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:33.685052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:33.729742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:33.730343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:33.730620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:33.730891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:33.731523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:33.733329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:57:33.733329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 06:57:33.733329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 06:57:33.734476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31c82c-a81a-457a-9d78-2b6a2566acd3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:33.753404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 729/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 9 06:57:33 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:57:35.746595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c026d6-81bc-411c-820d-d7471e5d3d12 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:35.747166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c026d6-81bc-411c-820d-d7471e5d3d12 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:35.747166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c026d6-81bc-411c-820d-d7471e5d3d12 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:35.748553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1479] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:35.754060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6315f16-f832-469f-b60a-6eca9d41ca0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:35.754495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6315f16-f832-469f-b60a-6eca9d41ca0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:35.754608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6315f16-f832-469f-b60a-6eca9d41ca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:35.944591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e6315f16-f832-469f-b60a-6eca9d41ca0d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:35.945814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e6315f16-f832-469f-b60a-6eca9d41ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:36.040597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 730/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:36.083881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c026d6-81bc-411c-820d-d7471e5d3d12 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:36.085459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66c026d6-81bc-411c-820d-d7471e5d3d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:36.095777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:36.230842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c026d6-81bc-411c-820d-d7471e5d3d12 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:36.309118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:36.342669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:36.343279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:36.343567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:36.343872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:36.345483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:36.434311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:36.435961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabd587-5331-4f6e-be85-d592f3a43cd7 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:36.453468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:38.611364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cd2d3-8351-4ff1-aeae-be6d0a9fcde1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:38.611944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cd2d3-8351-4ff1-aeae-be6d0a9fcde1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:38.611944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cd2d3-8351-4ff1-aeae-be6d0a9fcde1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:38.613284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 731/1482] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:38.618623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742c5c-2f8c-47d0-b542-d09a92475be6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:38.619125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742c5c-2f8c-47d0-b542-d09a92475be6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:38.619244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742c5c-2f8c-47d0-b542-d09a92475be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:38.684333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-30742c5c-2f8c-47d0-b542-d09a92475be6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:38.685471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30742c5c-2f8c-47d0-b542-d09a92475be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:38.733218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:38.786460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:38.823432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cd2d3-8351-4ff1-aeae-be6d0a9fcde1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:38.824969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f5cd2d3-8351-4ff1-aeae-be6d0a9fcde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:38.848714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:38.928062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cd2d3-8351-4ff1-aeae-be6d0a9fcde1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:39.001108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:39.001638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:39.001917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:39.002188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:39.003816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:39.075803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:39.111668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:39.113091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee80e9e-7982-4af4-835c-ec988793f10f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:39.130916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 732/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:41.206065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23666d65-05a8-468a-82b5-f6bf3c203653 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:41.206663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23666d65-05a8-468a-82b5-f6bf3c203653 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:41.206663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23666d65-05a8-468a-82b5-f6bf3c203653 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:41.207989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1485] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:41.213635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb159c9-bfd1-45b3-b556-2a7a762b8886 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:41.214125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb159c9-bfd1-45b3-b556-2a7a762b8886 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:41.214241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb159c9-bfd1-45b3-b556-2a7a762b8886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:41.281927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-feb159c9-bfd1-45b3-b556-2a7a762b8886 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:41.283152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-feb159c9-bfd1-45b3-b556-2a7a762b8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:41.381903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 733/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:41.420833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23666d65-05a8-468a-82b5-f6bf3c203653 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:41.422379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23666d65-05a8-468a-82b5-f6bf3c203653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:41.524306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23666d65-05a8-468a-82b5-f6bf3c203653 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:41.598544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:41.600596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:41.601318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:41.601845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:41.602187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:41.604798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:41.717073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:41.718492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abe023-84dd-4637-88e4-650d173de1eb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:41.736763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:43.837799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01b2ae-f347-471f-b9a9-a9960fe091aa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:43.838402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01b2ae-f347-471f-b9a9-a9960fe091aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:43.838402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01b2ae-f347-471f-b9a9-a9960fe091aa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:43.839762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 734/1488] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:43.845147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0895-4f81-4b37-8377-6ba34718181c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:43.845604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0895-4f81-4b37-8377-6ba34718181c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:43.845706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0895-4f81-4b37-8377-6ba34718181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:43.912389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b6b0895-4f81-4b37-8377-6ba34718181c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:43.913623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b6b0895-4f81-4b37-8377-6ba34718181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:43.965781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:44.010464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 06:57:44.050601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01b2ae-f347-471f-b9a9-a9960fe091aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:44.052156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a01b2ae-f347-471f-b9a9-a9960fe091aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:44.087028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:44.154612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01b2ae-f347-471f-b9a9-a9960fe091aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:44.226727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:44.227328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:44.227621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:44.227892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:44.229646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:44.307815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:44.337074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:44.338173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6363a75-5513-4e96-94f3-bddcac41fa3a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:44.356412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 735/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:46.534159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc623470-9700-4c18-9e0d-63807157b392 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:46.534711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc623470-9700-4c18-9e0d-63807157b392 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:46.534711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc623470-9700-4c18-9e0d-63807157b392 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:46.536130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1491] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 9 06:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:57:46.542211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ef34f-38bc-4837-b9aa-85a932b51237 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:46.542625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ef34f-38bc-4837-b9aa-85a932b51237 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:46.542765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ef34f-38bc-4837-b9aa-85a932b51237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:46.608928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d13ef34f-38bc-4837-b9aa-85a932b51237 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:57:46.610158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d13ef34f-38bc-4837-b9aa-85a932b51237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:46.708676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 736/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 9 06:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:57:46.749651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc623470-9700-4c18-9e0d-63807157b392 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:57:46.751565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc623470-9700-4c18-9e0d-63807157b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:46.857275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc623470-9700-4c18-9e0d-63807157b392 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:57:46.929995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:57:46.930599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:57:46.930877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:57:46.931148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:57:46.933665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:57:46.942640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:47.042450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:57:47.043873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69d3bc-ee82-46e5-83de-18fb10edd02b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:57:47.062212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:57:56.706682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=82 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:57:57.060064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=50 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:03.805405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:03.806004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:03.806004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:03.880555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:59:03.881822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:03.976873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 737/1494] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 06:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:59:03.985667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:03.986156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:03.986260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:04.061513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:59:04.062685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:04.157190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1495] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 06:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:59:04.161868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:04.163093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:04.174615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ee64-d0d8-4520-92cc-cb7664979f1a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:04.234874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:04.270057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:04.270656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:04.270933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:04.271205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:04.271869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 06:59:04.272178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:04.273178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:04.273178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 06:59:04.273178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 06:59:04.274054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:04.285700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 738/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 06:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 06:59:04.294640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:04.303503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4af6c9-37b3-42b6-a8a4-df5561d09ea7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:04.365818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:04.399459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:04.400861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:04.401248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:04.401527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:04.402145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:04.403901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:04.403901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 06:59:04.403901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 06:59:04.405095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:04.459994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 06:59:04 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:59:04.464322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:04.465885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:04.498890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eeef2-a4ab-4d4b-939d-801b9095a83d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:04.595618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:04.596226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:04.596502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:04.596771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:04.597390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:04.598392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:04.598392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 06:59:04.598392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 06:59:04.599585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:04.642152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 739/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 06:59:04 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 06:59:04.646364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:04.647912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:04.710088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b447a4b-612c-4c8f-b263-d2db0c4cb966 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:04.797719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:04.809674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:04.810287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:04.810569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:04.810843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:04.811494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:04.812528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:04.812528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 06:59:04.812528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 06:59:04.813456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:04.825600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 06:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:59:04.829628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:04.831137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:04.843222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c0605-18fd-4f46-a6e3-57e4e2a4a1b0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:04.869766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:04.938626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:04.939156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:04.939435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:04.939807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:04.940453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:04.941455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:04.941455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 06:59:04.941455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 06:59:04.942335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a29987-5564-4812-bd2f-81b6a8bdb1ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:04.954339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 740/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 06:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 06:59:04.958550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:04.960125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:04.993144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157482c-b305-4f5b-b890-f7fb0235cd66 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:05.088291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:05.088947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:05.089233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:05.089507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] PATH_INFO: `/v3/projects/1a6c7326e366471f982432bddd2cc8a8/users/93342f3e4e184e0789552a1d1fa8a5ae/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:05.111500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6c7326e366471f982432bddd2cc8a8, user_id=93342f3e4e184e0789552a1d1fa8a5ae, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:05.112545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:05.112545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 06:59:05.112545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 06:59:05.114477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e58603-1353-4e02-b348-25ff8e4d6a68 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:05.138700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 06:59:04 2022] PUT /identity/v3/projects/1a6c7326e366471f982432bddd2cc8a8/users/93342f3e4e184e0789552a1d1fa8a5ae/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:59:05.150924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d846a8-296e-42c3-8739-f19a9981012d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:05.151211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d846a8-296e-42c3-8739-f19a9981012d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:05.151485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d846a8-296e-42c3-8739-f19a9981012d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:05.208119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e1d846a8-296e-42c3-8739-f19a9981012d None None] MFA Rules not processed for user `93342f3e4e184e0789552a1d1fa8a5ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:59:05.209268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e1d846a8-296e-42c3-8739-f19a9981012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:05.286275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 741/1502] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 06:59:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:59:05.300137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa5903-629f-4049-9fb4-f10fec46a756 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:05.300423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa5903-629f-4049-9fb4-f10fec46a756 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:05.300698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa5903-629f-4049-9fb4-f10fec46a756 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:05.302039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1503] 10.222.0.30 () {54 vars in 964 bytes} [Tue Aug 9 06:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:59:05.306901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5456-1420-4029-aad7-7985aff56240 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:05.307184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5456-1420-4029-aad7-7985aff56240 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:05.307460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5456-1420-4029-aad7-7985aff56240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:05.338539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:05.383242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7f9b5456-1420-4029-aad7-7985aff56240 None None] MFA Rules not processed for user `6c7d784c96b044ef8b383767e8c1cb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:59:05.384485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f9b5456-1420-4029-aad7-7985aff56240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:05.458844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 742/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Aug 9 06:59:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:59:05.467186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa5903-629f-4049-9fb4-f10fec46a756 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:05.468728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16aa5903-629f-4049-9fb4-f10fec46a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:05.517710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:05.572522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa5903-629f-4049-9fb4-f10fec46a756 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:05.644049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:05.644646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:05.644921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:05.645187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:05.646771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:05.742129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:05.743463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:05.747012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:05.762539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 763/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:59:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:59:05.807387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9538ff-b9e2-40a3-b148-ce6c41e745e3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:05.807716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9538ff-b9e2-40a3-b148-ce6c41e745e3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:05.807998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9538ff-b9e2-40a3-b148-ce6c41e745e3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:05.809311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 743/1506] 10.222.0.30 () {52 vars in 830 bytes} [Tue Aug 9 06:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 06:59:05.812765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:05.813970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:05.913213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f01c-8670-4bd6-83df-e4552c6920da service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:06.005947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:06.006543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:06.006766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:06.007040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] PATH_INFO: `/v3/users/93342f3e4e184e0789552a1d1fa8a5ae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:06.019197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] RBAC: Authorizing `identity:delete_user(user_id=93342f3e4e184e0789552a1d1fa8a5ae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:06.020244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:06.020244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 06:59:06.020244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 06:59:06.021459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:06.031487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:06.086647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93342f3e4e184e0789552a1d1fa8a5ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:06.096112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93342f3e4e184e0789552a1d1fa8a5ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:06.110405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93342f3e4e184e0789552a1d1fa8a5ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:06.110844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93342f3e4e184e0789552a1d1fa8a5ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:06.133219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93342f3e4e184e0789552a1d1fa8a5ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:06.138819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 764/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 06:59:05 2022] DELETE /identity/v3/users/93342f3e4e184e0789552a1d1fa8a5ae => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:59:06.143219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:06.143726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:06.143830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:06.202005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 06:59:06.203221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:06.276476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:06.291667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 744/1508] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 06:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 06:59:06.301230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:06.302497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:06.401621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4088b-6932-495d-abc3-e029d4dac8da admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:06.455831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:06.477182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:06.477791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:06.478780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:06.479059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:06.480688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:06.595505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:06.596900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aca3ae-3027-44e1-9e31-614d6506c981 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:06.615394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 765/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 06:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 06:59:06.966670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 06:59:06.968031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 06:59:06.980666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c7649-3adb-4c86-982e-d36d303aa090 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 06:59:07.085426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 06:59:07.086061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 06:59:07.086340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 06:59:07.086648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] PATH_INFO: `/v3/projects/1a6c7326e366471f982432bddd2cc8a8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 06:59:07.097181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6c7326e366471f982432bddd2cc8a8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 06:59:07.098208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 06:59:07.098208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 06:59:07.098208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 06:59:07.099450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 06:59:07.157021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6c7326e366471f982432bddd2cc8a8', 'request_id': 'req-8a865554-ee5b-4a30-9582-e5e66a67c65d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:07.162964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6c7326e366471f982432bddd2cc8a8', 'request_id': 'req-8a865554-ee5b-4a30-9582-e5e66a67c65d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:07.176933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6c7326e366471f982432bddd2cc8a8', 'request_id': 'req-8a865554-ee5b-4a30-9582-e5e66a67c65d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:07.202203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] The token cache is being invalidate because project 1a6c7326e366471f982432bddd2cc8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 06:59:07.202496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:07.203016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8a865554-ee5b-4a30-9582-e5e66a67c65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 06:59:07.204358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 745/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 06:59:06 2022] DELETE /identity/v3/projects/1a6c7326e366471f982432bddd2cc8a8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 06:59:16.615860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=62 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 06:59:17.200225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:00:30.322007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06167fa1-4b0b-43e0-a24e-0991671b48de None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:30.322622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06167fa1-4b0b-43e0-a24e-0991671b48de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:30.322622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06167fa1-4b0b-43e0-a24e-0991671b48de None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:30.323930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 07:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:30.329329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d5f2-d959-468e-83cb-ca05a00a9c11 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:30.329777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d5f2-d959-468e-83cb-ca05a00a9c11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:30.329887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d5f2-d959-468e-83cb-ca05a00a9c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:30.412230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a2c7d5f2-d959-468e-83cb-ca05a00a9c11 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:30.413429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a2c7d5f2-d959-468e-83cb-ca05a00a9c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:30.515779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 07:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:30.673601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06167fa1-4b0b-43e0-a24e-0991671b48de None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:30.675736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-06167fa1-4b0b-43e0-a24e-0991671b48de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:30.799150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06167fa1-4b0b-43e0-a24e-0991671b48de None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:30.804533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8959ab-5caf-4513-aab2-2c19c5e8d00b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:30.805133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8959ab-5caf-4513-aab2-2c19c5e8d00b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:30.805409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8959ab-5caf-4513-aab2-2c19c5e8d00b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:30.805683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8959ab-5caf-4513-aab2-2c19c5e8d00b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:30.814496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8959ab-5caf-4513-aab2-2c19c5e8d00b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:30.816089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8959ab-5caf-4513-aab2-2c19c5e8d00b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:30.825589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 767/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:31.654655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f80a-2e54-4b09-9936-af5b4a8ba726 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:31.655225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f80a-2e54-4b09-9936-af5b4a8ba726 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:31.655225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f80a-2e54-4b09-9936-af5b4a8ba726 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:31.656602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 07:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:31.661413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb9a4d-04ee-4881-8484-eb86f3d42c9b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:31.661838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb9a4d-04ee-4881-8484-eb86f3d42c9b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:31.661967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb9a4d-04ee-4881-8484-eb86f3d42c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:31.681063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2cb9a4d-04ee-4881-8484-eb86f3d42c9b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:31.682259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2cb9a4d-04ee-4881-8484-eb86f3d42c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:31.690744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 07:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:31.842837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5f80a-2e54-4b09-9936-af5b4a8ba726 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:31.857090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5f80a-2e54-4b09-9936-af5b4a8ba726 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:31.862809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e0d59-6eae-496d-839a-12d417bceed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:31.863409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e0d59-6eae-496d-839a-12d417bceed3 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:31.863720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e0d59-6eae-496d-839a-12d417bceed3 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:31.863997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e0d59-6eae-496d-839a-12d417bceed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:31.872873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e0d59-6eae-496d-839a-12d417bceed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:31.874459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e0d59-6eae-496d-839a-12d417bceed3 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:31.884108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 748/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:32.296343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95191-ee5e-439c-9715-04c9ea2dc23e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:32.296561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95191-ee5e-439c-9715-04c9ea2dc23e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:32.296809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95191-ee5e-439c-9715-04c9ea2dc23e None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:32.298130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 769/1517] 10.222.0.26 () {54 vars in 925 bytes} [Tue Aug 9 07:00:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:00:32.304104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579f11-c742-46ad-8237-2597d78c1a42 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:32.304463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579f11-c742-46ad-8237-2597d78c1a42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:32.304744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579f11-c742-46ad-8237-2597d78c1a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:32.365755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c579f11-c742-46ad-8237-2597d78c1a42 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:32.366965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c579f11-c742-46ad-8237-2597d78c1a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:32.384208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:00:32.439184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 749/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Aug 9 07:00:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:36.236930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69342d29-85e1-4a9f-b532-4e88465b1691 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:36.237510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69342d29-85e1-4a9f-b532-4e88465b1691 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:36.237510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69342d29-85e1-4a9f-b532-4e88465b1691 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:36.238803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 770/1519] 10.222.0.30 () {54 vars in 961 bytes} [Tue Aug 9 07:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:36.244004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa729-bfe1-4d16-b19b-7167ee4eb0d5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:36.244438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa729-bfe1-4d16-b19b-7167ee4eb0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:36.244555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa729-bfe1-4d16-b19b-7167ee4eb0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:36.264688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-59afa729-bfe1-4d16-b19b-7167ee4eb0d5 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:36.265868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59afa729-bfe1-4d16-b19b-7167ee4eb0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:36.274591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 750/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Aug 9 07:00:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:36.282904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69342d29-85e1-4a9f-b532-4e88465b1691 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:36.296769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69342d29-85e1-4a9f-b532-4e88465b1691 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:36.301665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd6aa-064b-4711-8a4f-34951125d198 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:36.302260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd6aa-064b-4711-8a4f-34951125d198 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:36.302543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd6aa-064b-4711-8a4f-34951125d198 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:36.302819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd6aa-064b-4711-8a4f-34951125d198 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:36.311340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bd6aa-064b-4711-8a4f-34951125d198 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:36.312947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bd6aa-064b-4711-8a4f-34951125d198 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:36.322444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 771/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:43.125424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961313-d880-4470-8812-b0d940fedf5c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:43.125952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961313-d880-4470-8812-b0d940fedf5c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:43.125952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961313-d880-4470-8812-b0d940fedf5c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:43.127203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 07:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:43.132171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836542-f077-4c7d-a270-5d75d062bb38 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:43.132595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836542-f077-4c7d-a270-5d75d062bb38 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:43.132730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836542-f077-4c7d-a270-5d75d062bb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:43.152142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-69836542-f077-4c7d-a270-5d75d062bb38 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:43.154012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69836542-f077-4c7d-a270-5d75d062bb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:43.161436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 07:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:43.316252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb961313-d880-4470-8812-b0d940fedf5c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:43.331121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb961313-d880-4470-8812-b0d940fedf5c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:43.336562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235161b-c173-4f4d-874a-c0d7df825cdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:43.337155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235161b-c173-4f4d-874a-c0d7df825cdd service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:43.337434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235161b-c173-4f4d-874a-c0d7df825cdd service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:43.337706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235161b-c173-4f4d-874a-c0d7df825cdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:43.345653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235161b-c173-4f4d-874a-c0d7df825cdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:43.347230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235161b-c173-4f4d-874a-c0d7df825cdd service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:43.357307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 752/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:44.190448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1203-9ef9-4888-8e89-9d59716cc9ce None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:44.190998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1203-9ef9-4888-8e89-9d59716cc9ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:44.190998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1203-9ef9-4888-8e89-9d59716cc9ce None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:44.192384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 07:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:44.197308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522806-433a-4188-bf4f-8e096cc8941a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:44.197711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522806-433a-4188-bf4f-8e096cc8941a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:44.197847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522806-433a-4188-bf4f-8e096cc8941a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:44.218105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e0522806-433a-4188-bf4f-8e096cc8941a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:44.219388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e0522806-433a-4188-bf4f-8e096cc8941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:44.227133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 07:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:44.379616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1203-9ef9-4888-8e89-9d59716cc9ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:44.394221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1203-9ef9-4888-8e89-9d59716cc9ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:44.399568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2d210-7aa0-4e15-af76-633b0ce306fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:44.400188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2d210-7aa0-4e15-af76-633b0ce306fd service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:44.400466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2d210-7aa0-4e15-af76-633b0ce306fd service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:44.400737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2d210-7aa0-4e15-af76-633b0ce306fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:44.409331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2d210-7aa0-4e15-af76-633b0ce306fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:44.410888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2d210-7aa0-4e15-af76-633b0ce306fd service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:44.420292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 774/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:48.261025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd372f3-f08f-462b-a273-41cbf94bec80 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:48.261662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd372f3-f08f-462b-a273-41cbf94bec80 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:48.261662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd372f3-f08f-462b-a273-41cbf94bec80 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:48.263656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 07:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:48.268778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1910db9-cc4d-45ae-9dc4-fdb7a316801f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:48.269217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1910db9-cc4d-45ae-9dc4-fdb7a316801f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:48.269328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1910db9-cc4d-45ae-9dc4-fdb7a316801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:48.288435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c1910db9-cc4d-45ae-9dc4-fdb7a316801f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:48.289668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1910db9-cc4d-45ae-9dc4-fdb7a316801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:48.297436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 07:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:48.369014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd372f3-f08f-462b-a273-41cbf94bec80 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:48.370731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-edd372f3-f08f-462b-a273-41cbf94bec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:48.448835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd372f3-f08f-462b-a273-41cbf94bec80 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:48.453014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff60e6-0134-44c4-9d56-dc2c81e88808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:48.453613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff60e6-0134-44c4-9d56-dc2c81e88808 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:48.453892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff60e6-0134-44c4-9d56-dc2c81e88808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:48.454163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff60e6-0134-44c4-9d56-dc2c81e88808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:48.463129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff60e6-0134-44c4-9d56-dc2c81e88808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:48.464574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff60e6-0134-44c4-9d56-dc2c81e88808 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:48.474572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 755/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:50.485838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401f3d7-1066-4b31-9f9e-f8df67f12b24 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:50.486398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401f3d7-1066-4b31-9f9e-f8df67f12b24 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:50.486398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401f3d7-1066-4b31-9f9e-f8df67f12b24 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:50.487773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 07:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:50.493012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec64d5-b46a-4ed6-9756-1f71d970f68e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:50.493413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec64d5-b46a-4ed6-9756-1f71d970f68e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:50.493563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec64d5-b46a-4ed6-9756-1f71d970f68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:50.514824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cdec64d5-b46a-4ed6-9756-1f71d970f68e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:50.516065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cdec64d5-b46a-4ed6-9756-1f71d970f68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:50.523916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 07:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:50.560940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401f3d7-1066-4b31-9f9e-f8df67f12b24 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:50.572184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401f3d7-1066-4b31-9f9e-f8df67f12b24 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:50.576707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884560a4-22cc-4d35-97a3-1f73aa658569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:50.577293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884560a4-22cc-4d35-97a3-1f73aa658569 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:50.577570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884560a4-22cc-4d35-97a3-1f73aa658569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:50.577839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884560a4-22cc-4d35-97a3-1f73aa658569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:50.586524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884560a4-22cc-4d35-97a3-1f73aa658569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:50.587965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884560a4-22cc-4d35-97a3-1f73aa658569 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:50.597183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 777/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:54.071165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817164e-9c51-4a26-9b5b-64c0d784365e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:54.071745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817164e-9c51-4a26-9b5b-64c0d784365e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:54.071745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817164e-9c51-4a26-9b5b-64c0d784365e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:54.073108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 07:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:54.078872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542456-8273-436e-8293-287fbe97da8f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:54.079294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542456-8273-436e-8293-287fbe97da8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:54.079410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542456-8273-436e-8293-287fbe97da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:54.099083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b542456-8273-436e-8293-287fbe97da8f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:54.100333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b542456-8273-436e-8293-287fbe97da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:54.108305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:54.411278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817164e-9c51-4a26-9b5b-64c0d784365e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:54.412974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2817164e-9c51-4a26-9b5b-64c0d784365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:54.478671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:00:54.492282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817164e-9c51-4a26-9b5b-64c0d784365e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:54.495887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2e9c9-40fc-47c7-89e1-66cdf3c38d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:54.496498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2e9c9-40fc-47c7-89e1-66cdf3c38d09 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:54.497325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2e9c9-40fc-47c7-89e1-66cdf3c38d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:54.497675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2e9c9-40fc-47c7-89e1-66cdf3c38d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:54.506824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2e9c9-40fc-47c7-89e1-66cdf3c38d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:54.508286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2e9c9-40fc-47c7-89e1-66cdf3c38d09 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:54.518317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 758/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:00:56.545448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c6a9a-1e52-45cd-8eef-16860ca529da None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:56.545867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c6a9a-1e52-45cd-8eef-16860ca529da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:56.546145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c6a9a-1e52-45cd-8eef-16860ca529da None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:56.547336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 07:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:00:56.552628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b3a5d-8700-49ea-8b2e-25037c4b99cd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:56.553021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b3a5d-8700-49ea-8b2e-25037c4b99cd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:56.553177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b3a5d-8700-49ea-8b2e-25037c4b99cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:56.573025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-489b3a5d-8700-49ea-8b2e-25037c4b99cd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:00:56.574260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-489b3a5d-8700-49ea-8b2e-25037c4b99cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:56.582313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:00:56.885964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c6a9a-1e52-45cd-8eef-16860ca529da None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:00:56.888294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c61c6a9a-1e52-45cd-8eef-16860ca529da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:00:56.895868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:00:56.902250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c6a9a-1e52-45cd-8eef-16860ca529da None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:00:56.906442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ea54-3365-4335-8068-d72d8e8b6b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:00:56.907041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ea54-3365-4335-8068-d72d8e8b6b1e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:00:56.907317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ea54-3365-4335-8068-d72d8e8b6b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:00:56.907587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ea54-3365-4335-8068-d72d8e8b6b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:00:56.916388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ea54-3365-4335-8068-d72d8e8b6b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:00:56.917790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ea54-3365-4335-8068-d72d8e8b6b1e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:00:56.926914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 780/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:01:06.567027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=11 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:01:06.922869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:19.311203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.312659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.312659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.340990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.342264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.360716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 760/1540] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:02:19.370562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bb06a-5c35-4615-82ef-8f531ad4d5e5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.371216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bb06a-5c35-4615-82ef-8f531ad4d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.371216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bb06a-5c35-4615-82ef-8f531ad4d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.404160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-640bb06a-5c35-4615-82ef-8f531ad4d5e5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.405388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-640bb06a-5c35-4615-82ef-8f531ad4d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.425435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 781/1541] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.433627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:19.449019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323b003-105e-4a06-9f18-ae9991d8458a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:19.460074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:19.460700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.460979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.461250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.461905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:19.462205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:19.467993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:19.481075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969c82-6b9c-4a19-b57a-bef08c61a90c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.481403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969c82-6b9c-4a19-b57a-bef08c61a90c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.481644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969c82-6b9c-4a19-b57a-bef08c61a90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.481898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 761/1542] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:19.489773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:19.500517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71d1c2-349b-4430-ae79-6b89918dc0f3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:19.513345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d8969c82-6b9c-4a19-b57a-bef08c61a90c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.515400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d8969c82-6b9c-4a19-b57a-bef08c61a90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.517476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:19.518118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.518398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.518666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.519303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:19.520632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 07:02:19.520632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 07:02:19.520632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 07:02:19.521829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327d9aa-d9e9-4c30-83f1-5a66a324b48c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:19.530465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 782/1543] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:02:19.539238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.539531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.539847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.569623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.570907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.585696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 762/1544] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:19.587229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.587523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.587856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.599702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 783/1545] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:02:19.600895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:19.617279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28003c-7dc5-4cb3-9f9a-7d8a20937ca8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:19.620083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.621662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.705516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:19.706372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.706656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.706990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.707731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:19.708652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 763/1546] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.709149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 07:02:19.709149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 07:02:19.709149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 07:02:19.709780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:19.710721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b174c-80f0-4b55-8e94-be06ce299b2e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:19.720175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e589a-cca0-4357-b008-d0a43d9c7a9b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:19.730143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:19.730775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.731101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.731376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.732073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:19.732377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:19.734263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd09184-1247-4186-b7d6-a2e4f654401d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:19.747673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 784/1547] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:19.750058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 764/1548] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:19.751646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79dbfa-72ea-4fca-981d-88c71cd56993 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.751938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79dbfa-72ea-4fca-981d-88c71cd56993 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.752211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79dbfa-72ea-4fca-981d-88c71cd56993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.753319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9775a-b011-4cfc-80d9-2b5c3b83e1b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.753621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9775a-b011-4cfc-80d9-2b5c3b83e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.753915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9775a-b011-4cfc-80d9-2b5c3b83e1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.772847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4c79dbfa-72ea-4fca-981d-88c71cd56993 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.774060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4c79dbfa-72ea-4fca-981d-88c71cd56993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.774451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42c9775a-b011-4cfc-80d9-2b5c3b83e1b1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.775672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42c9775a-b011-4cfc-80d9-2b5c3b83e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.782121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 765/1549] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.783290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbd3da-b937-432a-8801-52664b8151f3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.783569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbd3da-b937-432a-8801-52664b8151f3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.783866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbd3da-b937-432a-8801-52664b8151f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.789406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 785/1550] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.794000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeec185-036b-4bb8-ac18-62ef30cd7f81 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.794289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeec185-036b-4bb8-ac18-62ef30cd7f81 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.794569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeec185-036b-4bb8-ac18-62ef30cd7f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.802960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5ecbd3da-b937-432a-8801-52664b8151f3 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.804182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5ecbd3da-b937-432a-8801-52664b8151f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.814090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 766/1551] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.814795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3aeec185-036b-4bb8-ac18-62ef30cd7f81 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.815320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.815782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.816067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.816181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3aeec185-036b-4bb8-ac18-62ef30cd7f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.824774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 786/1552] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.825982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.826269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.826541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.836447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.837666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.847031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.848260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.848760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 767/1553] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.850403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:19.859649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 787/1554] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.859649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:19.861190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade18f4-25f4-4d89-b6a0-c9b46e1d7f12 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:19.868524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32b085-4958-403c-be84-4747e6fb8bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:19.869133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32b085-4958-403c-be84-4747e6fb8bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.869461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32b085-4958-403c-be84-4747e6fb8bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.869880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32b085-4958-403c-be84-4747e6fb8bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.869990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44ea2-de37-41a9-a5f6-1a0a5ff91e6d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:19.870554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32b085-4958-403c-be84-4747e6fb8bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:19.872173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32b085-4958-403c-be84-4747e6fb8bb2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:19.874819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b637543-4887-4613-ad32-2e22adcddde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:19.875415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637543-4887-4613-ad32-2e22adcddde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.875734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637543-4887-4613-ad32-2e22adcddde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.876006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637543-4887-4613-ad32-2e22adcddde3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.876661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637543-4887-4613-ad32-2e22adcddde3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:19.878526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637543-4887-4613-ad32-2e22adcddde3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:19.885766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 768/1555] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:19.885766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b166bc-d86c-4e44-a9f7-5b5d6e2df35d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.885766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b166bc-d86c-4e44-a9f7-5b5d6e2df35d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.885766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b166bc-d86c-4e44-a9f7-5b5d6e2df35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.905057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-99b166bc-d86c-4e44-a9f7-5b5d6e2df35d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.906268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-99b166bc-d86c-4e44-a9f7-5b5d6e2df35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.912158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 788/1556] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:19.913724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.914017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.914288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.918880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 769/1557] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:19.918880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9230-0b53-4ceb-8613-156e71626bbc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:19.918880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9230-0b53-4ceb-8613-156e71626bbc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:19.918880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9230-0b53-4ceb-8613-156e71626bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:19.934268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.935469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.936769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c08a9230-0b53-4ceb-8613-156e71626bbc None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:19.937931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c08a9230-0b53-4ceb-8613-156e71626bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:19.989683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:19.998961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 789/1558] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:20.003308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 770/1559] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:20.003456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.003550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d7b27-d4ff-46fa-a0b2-c75ccd7c82cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.003550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d7b27-d4ff-46fa-a0b2-c75ccd7c82cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.003550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d7b27-d4ff-46fa-a0b2-c75ccd7c82cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.012130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76704-8e26-44d0-b6d0-bdfc1f70937f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.016710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.017307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.017593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.017853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.018484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:20.018782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.019812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 07:02:20.019812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 07:02:20.019812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 07:02:20.020685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.021091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-694d7b27-d4ff-46fa-a0b2-c75ccd7c82cb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:20.022306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-694d7b27-d4ff-46fa-a0b2-c75ccd7c82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:20.030857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 771/1560] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:20.032069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.032352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.032624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.040477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 790/1561] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:20.040477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.051338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc11bd-9775-47ec-bf06-e9486dde6439 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.054663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:20.055890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:20.056337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.056968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.057250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.057520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.058168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.060283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.065601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 772/1562] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:20.066481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.077736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 791/1563] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:20.077736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.077981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6e7e-a187-46be-b3ff-d75332182f96 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.081656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.082256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.082397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:20.082532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.082809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.083423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.085440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.086117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0244-8b06-49b4-b6bc-bc769c073b5d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.092782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.093375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.093653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.093927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.094569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:20.094868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.096495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.110028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 792/1564] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:20.110799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.117572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 773/1565] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 283 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.118413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.126489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6cbb2-aa41-4e54-8ee3-4958d3039f09 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.129946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffed1c-105d-4ecd-a20d-73079c2432e5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.131482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.132123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.132401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.132796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.133468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:20.133772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.134725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.136020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.136327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.136433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.136594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.137239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:20.137542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.139125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.150063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 793/1566] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:20.152253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 774/1567] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:20.153058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.158806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.169569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02cd35-b3c1-4592-851f-3282d7305af4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.169882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d350-3e73-48fa-add6-7d71e64a3c28 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.174088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.174683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.174811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.174956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.175223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.175362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.175676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.175861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.175966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.176577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:20.176879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.177735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.178478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.191197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 794/1568] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:20.192030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.204177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960d59-91e8-49af-964a-710d7efc293b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.208777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.209369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.209647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.209920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.210553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:20.210776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 775/1569] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.210876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.211580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.212586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.226214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 795/1570] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:20.226854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.238753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeeb8c-52d5-4357-8a38-3de763b23885 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.282922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4912759-ae53-4112-b502-3b1d5e28a82e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.287497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.288182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.288459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.288730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] PATH_INFO: `/v3/projects/f38c6aa37b9643bbbe5bb7fd3efde4fd/users/cc885026afe041cca086367bb67d8a9b/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.295757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.296465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.296744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.297018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.297637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.299536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.303499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38c6aa37b9643bbbe5bb7fd3efde4fd, user_id=cc885026afe041cca086367bb67d8a9b, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.304931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 07:02:20.304931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 07:02:20.304931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 07:02:20.307944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.321451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 776/1571] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:20 2022] PUT /identity/v3/projects/f38c6aa37b9643bbbe5bb7fd3efde4fd/users/cc885026afe041cca086367bb67d8a9b/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:20.322337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.328498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 796/1572] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.328498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.383837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443d32d-c11b-45b3-b52e-b17cc9446425 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.388556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.389165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.389443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.389718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.389897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336e293-c950-49e2-863c-f671603b0430 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.390415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.392098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.394629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.395227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.395503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.395804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.396419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.398290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.404052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 777/1573] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:20.405093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.415292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7027d-e650-4246-9bad-381b664f6505 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.421785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.422387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.422671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.422942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.423562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.425500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.428064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 797/1574] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.429029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.437879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:20.454615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 778/1575] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.455440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.456751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:20.498436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ea767-3a56-4cdc-bfc3-355f9396e4c1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.518708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9ac67-e7d6-4d62-b4ff-6418a4176aa8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.523487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.524525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.524809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.525138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.526394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.528863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.551568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 779/1576] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.552401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.558030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402971c2-aedc-4cba-b7f6-4dca3f29c110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.558801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402971c2-aedc-4cba-b7f6-4dca3f29c110 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.559082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402971c2-aedc-4cba-b7f6-4dca3f29c110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.559357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402971c2-aedc-4cba-b7f6-4dca3f29c110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.560009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402971c2-aedc-4cba-b7f6-4dca3f29c110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.561894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402971c2-aedc-4cba-b7f6-4dca3f29c110 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.562380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de960-70d0-4a0d-9848-fdae74bfeb9e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.566952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.567540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.567853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.568153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.568732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.570609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.590869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 798/1577] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.591822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39055653-b50a-4b91-9089-377cd662853b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.592172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39055653-b50a-4b91-9089-377cd662853b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.592511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39055653-b50a-4b91-9089-377cd662853b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.602585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 780/1578] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.603468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.613730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39055653-b50a-4b91-9089-377cd662853b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:20.614919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39055653-b50a-4b91-9089-377cd662853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:20.665622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c53a-29d4-4355-9da6-f9def6759fde admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.670548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.671159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.671442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.671810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.672406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.674327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.675990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 799/1579] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:20.676903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39055653-b50a-4b91-9089-377cd662853b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.689586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39055653-b50a-4b91-9089-377cd662853b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.695194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.695820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.696098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.696372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.697010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.697134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 781/1580] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.698594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.699400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.712740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 800/1581] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:20.712923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3311c6d-bce3-4a14-b374-d361a10b1820 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.713041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.718031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.718630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.718907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.719178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.719825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.721742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.723259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a7196-0285-428c-9225-961263cd66a8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.727889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.728489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.728775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.729711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.730334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.732270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.745262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 782/1582] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.746182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.757839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 801/1583] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.757839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.758114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c3cda-6564-49c5-95a8-558d668d51a8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.762330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.762936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.763214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.763488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.764172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.765764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.767659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf95a72-26af-45ea-a6c6-ad099ac2ae64 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.772064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.772650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.772928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.773198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.773810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.776367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.776981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 783/1584] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:20.777757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.788813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d3797-1f16-4f87-8ce6-c92c734f3822 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.795314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.795949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.796227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.796514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.797107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.801628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 802/1585] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.801761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.801862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.810930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:20.821975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 784/1586] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:20 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:20.823540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.828446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:20.834463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed37e18-be9d-45fc-96e8-18437d987174 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.844677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.845220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.845667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.845939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.846582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.848642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.862092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 785/1587] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:20.862909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.866094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e6937-cd37-4992-9eba-725d41fb54ce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.869922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.870706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.871030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.871316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.872946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a31e9-947a-4fbe-a50c-0881cadff6fb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.877532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.878164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.878445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.878718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] PATH_INFO: `/v3/projects/9ba36b111393480a9ef36b94ec094614/users/2b4d2df023624527936d713ad7ab556c/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.880711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.882128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.892101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 803/1588] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:20.892967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.894312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba36b111393480a9ef36b94ec094614, user_id=2b4d2df023624527936d713ad7ab556c, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.897065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.902975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926ff6-74b6-47a1-bca7-b0459d859fb9 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.909052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:20.909619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:20.909997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:20.910338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:20.911041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:20.912720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:20.914139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 786/1589] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:20 2022] PUT /identity/v3/projects/9ba36b111393480a9ef36b94ec094614/users/2b4d2df023624527936d713ad7ab556c/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:20.916466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.927648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 804/1590] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:20.927648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:20.937983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551cfdd-640a-4053-b8c4-e8bd6599d463 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:20.993191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945e9a8-c56a-455f-90be-097e78640b65 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.001388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.002079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.002489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.002803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.003486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.005435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.006109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.007010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.007770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.008341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.008965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.011492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.020825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 787/1591] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.020825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.023200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 805/1592] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.023964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.033145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d787c-2b76-4d39-a9ac-78f75ff37082 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.039512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.040137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95730e7b-55be-4925-ab9d-78da07d8f3aa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.040265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.040420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.040699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.042169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.044049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.054623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca17fc0-9cb8-4fb6-a0d9-9145fd4190dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.055452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca17fc0-9cb8-4fb6-a0d9-9145fd4190dc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.055870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca17fc0-9cb8-4fb6-a0d9-9145fd4190dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.056272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca17fc0-9cb8-4fb6-a0d9-9145fd4190dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.060708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca17fc0-9cb8-4fb6-a0d9-9145fd4190dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.060857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 788/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.060857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.061069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca17fc0-9cb8-4fb6-a0d9-9145fd4190dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.070971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920455d-e6e6-4363-b481-47a13b17a54a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.072662 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 806/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.076800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.077020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.077195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.077598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.077914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.078263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.078607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.079325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.081033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.094811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 789/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.095486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.109368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] MFA Rules not processed for user `2b4d2df023624527936d713ad7ab556c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:21.110871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:21.124077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c061-6f07-4a50-8f4f-943228869a4a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.129641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.130493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.131076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.131206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.131984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.133794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.147054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 790/1596] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.149009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.159733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c5f24-7f54-4988-8dab-98f1e5c79433 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.165790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.166513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.166868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.167143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.167823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.169534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.183797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 791/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.184672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.187058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 807/1598] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:21.188046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.199138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d76b-5476-4931-a6f0-93b26fda9605 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.200352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcda10a-4731-44ba-9d8f-7ee71421d566 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.204375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.205008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.205287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.205448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.205577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] PATH_INFO: `/v3/projects/10c7e20a9669439fa43853dab6f9851c/users/0bb72c5f52fc498aa60eecba3bd5248a/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.206150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.206493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.206833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.207631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.209325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.221785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 808/1599] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.222819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.223695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c7e20a9669439fa43853dab6f9851c, user_id=0bb72c5f52fc498aa60eecba3bd5248a, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.227183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.229920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:21.234545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb58d6-8eba-4f72-8096-095a12829658 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.240823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.241483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.241785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.242017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.242653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.244294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.248621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 792/1600] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:21 2022] PUT /identity/v3/projects/10c7e20a9669439fa43853dab6f9851c/users/0bb72c5f52fc498aa60eecba3bd5248a/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:21.248621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.256316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 809/1601] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:21.260287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.272687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0601c-e55b-4919-a2a3-0d5cfe72ca79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.311843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:21.319092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a74eaf-20db-4329-9355-3e12cd64dc07 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.326083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.326754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.327028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.327310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] PATH_INFO: `/v3/projects/a1b8f338b40e4d7e9a6249084bcdaa73/users/20ef078fe46d418c95f35e1f0bdd2f2e/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.337531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.339107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.339690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.340151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] PATH_INFO: `/v3/projects/a20ca03aff6048049ace8e8bcafb6ee1/users/4fa7298e183f45e48e02419abecda282/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.343959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b8f338b40e4d7e9a6249084bcdaa73, user_id=20ef078fe46d418c95f35e1f0bdd2f2e, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.346700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.359015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a20ca03aff6048049ace8e8bcafb6ee1, user_id=4fa7298e183f45e48e02419abecda282, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.362385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.362521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 793/1602] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:21 2022] PUT /identity/v3/projects/a1b8f338b40e4d7e9a6249084bcdaa73/users/20ef078fe46d418c95f35e1f0bdd2f2e/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:21.363371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.378480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 810/1603] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:21 2022] PUT /identity/v3/projects/a20ca03aff6048049ace8e8bcafb6ee1/users/4fa7298e183f45e48e02419abecda282/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:21.380493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.451139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5b92d-0775-4802-a6df-c4241a92c244 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.461950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc28938-f0a7-497a-872f-1ad934fd61e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.467213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.467897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.468184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.468529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.526002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b585cbc-d51e-44d1-9bd5-d8ed19eae69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.526966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585cbc-d51e-44d1-9bd5-d8ed19eae69e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.527223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585cbc-d51e-44d1-9bd5-d8ed19eae69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.527674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585cbc-d51e-44d1-9bd5-d8ed19eae69e admin admin] PATH_INFO: `/v3/projects/b7ada2fd6ffa44df9345c13afb650496/users/4e528f53a4b94e0fab0d90e0dbe07216/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.536939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.538476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.544648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b585cbc-d51e-44d1-9bd5-d8ed19eae69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ada2fd6ffa44df9345c13afb650496, user_id=4e528f53a4b94e0fab0d90e0dbe07216, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.547547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b585cbc-d51e-44d1-9bd5-d8ed19eae69e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.550932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 811/1604] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:21.551856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.564489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847ca8f-4782-4169-bc85-83dfd2a57bce service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.574610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 794/1605] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:21 2022] PUT /identity/v3/projects/b7ada2fd6ffa44df9345c13afb650496/users/4e528f53a4b94e0fab0d90e0dbe07216/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:21.578527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.578995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.579384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.604330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:21.605611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:21.607361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:21.649222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed0dd1-4cb0-44be-8a8a-13b43d7a12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.650044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed0dd1-4cb0-44be-8a8a-13b43d7a12d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.650428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed0dd1-4cb0-44be-8a8a-13b43d7a12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.650870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed0dd1-4cb0-44be-8a8a-13b43d7a12d8 admin admin] PATH_INFO: `/v3/projects/9253508f842f4eb28b9971b1b1a47572/users/edf8e24e147149f1b05fb34cb55d154b/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.669551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed0dd1-4cb0-44be-8a8a-13b43d7a12d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9253508f842f4eb28b9971b1b1a47572, user_id=edf8e24e147149f1b05fb34cb55d154b, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.672379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed0dd1-4cb0-44be-8a8a-13b43d7a12d8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.691827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 795/1607] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:21.693141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.695699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 812/1607] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:21 2022] PUT /identity/v3/projects/9253508f842f4eb28b9971b1b1a47572/users/edf8e24e147149f1b05fb34cb55d154b/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:21.701543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87496877-0928-46b5-8f1a-0b2849b11e54 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.701838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87496877-0928-46b5-8f1a-0b2849b11e54 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.702125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87496877-0928-46b5-8f1a-0b2849b11e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.716737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2a587-e5b2-456d-ad13-bef1902925a5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:21.723183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:21.724787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-87496877-0928-46b5-8f1a-0b2849b11e54 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:21.726357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-87496877-0928-46b5-8f1a-0b2849b11e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:21.791900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4547-957c-49a1-b87c-5c28696aaca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:21.793035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4547-957c-49a1-b87c-5c28696aaca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.793649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4547-957c-49a1-b87c-5c28696aaca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.794162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4547-957c-49a1-b87c-5c28696aaca5 admin admin] PATH_INFO: `/v3/projects/517f3dedc5da431b87e398003506c06b/users/19da3c426c6e4c4a889ab5017804c92e/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.811064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 813/1608] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:21.811064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.811064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.811064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.823976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4547-957c-49a1-b87c-5c28696aaca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517f3dedc5da431b87e398003506c06b, user_id=19da3c426c6e4c4a889ab5017804c92e, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:21.832104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4547-957c-49a1-b87c-5c28696aaca5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:21.845400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] MFA Rules not processed for user `4fa7298e183f45e48e02419abecda282`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:21.846981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:21.864788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 796/1609] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:21 2022] PUT /identity/v3/projects/517f3dedc5da431b87e398003506c06b/users/19da3c426c6e4c4a889ab5017804c92e/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:21.865970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05443ad-3594-4414-beae-bc2712a8fd04 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.868514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05443ad-3594-4414-beae-bc2712a8fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.868903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05443ad-3594-4414-beae-bc2712a8fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.909554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d05443ad-3594-4414-beae-bc2712a8fd04 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:21.912742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d05443ad-3594-4414-beae-bc2712a8fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:21.985208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 797/1610] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:21.986556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5bdc9-a1d5-42ed-81bf-171844c76b96 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:21.986843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5bdc9-a1d5-42ed-81bf-171844c76b96 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:21.987116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5bdc9-a1d5-42ed-81bf-171844c76b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:21.993827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 814/1611] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:21.995315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:21.999307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:22.050558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aef5bdc9-a1d5-42ed-81bf-171844c76b96 None None] MFA Rules not processed for user `edf8e24e147149f1b05fb34cb55d154b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:22.052069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aef5bdc9-a1d5-42ed-81bf-171844c76b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:22.099172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:22.122359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956195f4-66bb-4c35-a749-2136611ae98f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.134131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.134776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.135159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.136009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.153005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.156605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.180714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 815/1612] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:22.182861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.205862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51211-8015-46af-91ac-6c0bc4b13dc7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.214616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfddbe-a985-4f11-8c5f-9baefa3eac19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.215296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 798/1613] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:22.217158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfddbe-a985-4f11-8c5f-9baefa3eac19 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.218000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfddbe-a985-4f11-8c5f-9baefa3eac19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.218588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfddbe-a985-4f11-8c5f-9baefa3eac19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.218855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.219301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.219923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.241599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfddbe-a985-4f11-8c5f-9baefa3eac19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.243155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfddbe-a985-4f11-8c5f-9baefa3eac19 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.250287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:22.251910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:22.263284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 799/1614] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:22.265214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.272803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 816/1615] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:22.281171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505287c-70fd-45ac-94c9-9d23766a1eba None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.281659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505287c-70fd-45ac-94c9-9d23766a1eba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.282300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505287c-70fd-45ac-94c9-9d23766a1eba None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.283931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 817/1616] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 9 07:02:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:22.288264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505287c-70fd-45ac-94c9-9d23766a1eba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.305309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bfc1b-a1c0-4fa4-8fb3-4705a3ea5a52 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.319383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82af0a-30a0-423d-8606-8009ded95516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.320390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82af0a-30a0-423d-8606-8009ded95516 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.320773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82af0a-30a0-423d-8606-8009ded95516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.321267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82af0a-30a0-423d-8606-8009ded95516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.351105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505287c-70fd-45ac-94c9-9d23766a1eba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.353567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82af0a-30a0-423d-8606-8009ded95516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.355819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82af0a-30a0-423d-8606-8009ded95516 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.367458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.368030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.368313 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.368588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.369220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.378508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 800/1617] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:22.380217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.381193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.385117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.385780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.403109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:22.447189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 818/1618] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:22 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:22.448337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.480672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:22.483306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:22.574841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57b5ed-c7e2-4b34-b50e-c8f75575b188 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.585172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.586032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.586474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.587202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.601223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 801/1619] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 9 07:02:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:22.602410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.711981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801907e-819d-457d-9c48-84c1af7ed5c4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.718274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.719248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.719671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.720146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.720926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.722378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.725002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.725404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.750167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 819/1620] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:22.751126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.770391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 802/1621] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:22 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:22.772601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.781909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04bdf1-f271-43d1-b7c3-c915019e6138 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.787164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023b1c9-899a-4dbf-8f05-5e5c073e4dc6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.793863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.794819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.795329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.795827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.796772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.797570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.798696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.798994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.801224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.801624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.819861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 803/1622] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:22.821953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.842594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63221f5c-a576-4cf7-8d8f-f7bb67fa1ff3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:22.849880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:22.850808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:22.851212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:22.853036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] PATH_INFO: `/v3/projects/ccfdb0bba51840ae88e3569f31187be7/users/e3e88d807ab946f59436a46ce32d7284/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:22.884779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfdb0bba51840ae88e3569f31187be7, user_id=e3e88d807ab946f59436a46ce32d7284, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.889225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.913196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 804/1623] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:22 2022] PUT /identity/v3/projects/ccfdb0bba51840ae88e3569f31187be7/users/e3e88d807ab946f59436a46ce32d7284/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:22.917912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:22.919515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:22.920764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:22.924757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:22.931235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 820/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:22.985547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:23.020033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469c93-ed15-4b56-93a8-44c411db1b0a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:23.026477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ca9a7-dc77-4b72-9f3a-039440b8f108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:23.027417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ca9a7-dc77-4b72-9f3a-039440b8f108 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.027938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ca9a7-dc77-4b72-9f3a-039440b8f108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.028468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ca9a7-dc77-4b72-9f3a-039440b8f108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.029628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ca9a7-dc77-4b72-9f3a-039440b8f108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:23.032121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ca9a7-dc77-4b72-9f3a-039440b8f108 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:23.048519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 805/1625] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:23.054030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:23.074105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cfd72-1c00-4963-9f2b-24b21379b25b service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:23.080973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0284389-97f6-43c4-9e15-0767e56a3298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:23.082429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0284389-97f6-43c4-9e15-0767e56a3298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.082805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0284389-97f6-43c4-9e15-0767e56a3298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.083361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0284389-97f6-43c4-9e15-0767e56a3298 admin admin] PATH_INFO: `/v3/projects/ccfdb0bba51840ae88e3569f31187be7/users/e3e88d807ab946f59436a46ce32d7284/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.086592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0284389-97f6-43c4-9e15-0767e56a3298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfdb0bba51840ae88e3569f31187be7, user_id=e3e88d807ab946f59436a46ce32d7284, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:23.091765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0284389-97f6-43c4-9e15-0767e56a3298 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:23.112017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 821/1626] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:23 2022] PUT /identity/v3/projects/ccfdb0bba51840ae88e3569f31187be7/users/e3e88d807ab946f59436a46ce32d7284/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:23.123479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.124197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.124346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.188211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] MFA Rules not processed for user `e3e88d807ab946f59436a46ce32d7284`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:23.190509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:23.346237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6530883-5170-423f-b5ed-49888982333e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.346570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6530883-5170-423f-b5ed-49888982333e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.346849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6530883-5170-423f-b5ed-49888982333e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.349248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 822/1627] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 9 07:02:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:23.359329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.360713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.361150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.361586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 806/1628] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:23.377957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:23.391527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:23.393459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:23.483009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4197-b59e-4001-8916-e2ef38d8a2a9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:23.489006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:23.491075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.491075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.491075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.505785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:23.508031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:23.558316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 807/1629] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:23.608825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 823/1630] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:02:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:23.619481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:23.637577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a9334-ed69-45e4-8e72-a00ac2a8421f service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:23.647662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a4d6-9771-42ab-9aab-e87fba0c7814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:23.648723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a4d6-9771-42ab-9aab-e87fba0c7814 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.649790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a4d6-9771-42ab-9aab-e87fba0c7814 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.650258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a4d6-9771-42ab-9aab-e87fba0c7814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:23.721091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a4d6-9771-42ab-9aab-e87fba0c7814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:23.721091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a4d6-9771-42ab-9aab-e87fba0c7814 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:23.738264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 808/1631] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:23.888860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:23.915912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab56df-4dab-484f-96ef-4ebb19876bd0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:23.926917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf47e1-45e3-425b-abd8-6cfe2d25125f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:23.927696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf47e1-45e3-425b-abd8-6cfe2d25125f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:23.927987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf47e1-45e3-425b-abd8-6cfe2d25125f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:23.928273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf47e1-45e3-425b-abd8-6cfe2d25125f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:24.076271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf47e1-45e3-425b-abd8-6cfe2d25125f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:24.078665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf47e1-45e3-425b-abd8-6cfe2d25125f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:24.104904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 824/1632] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:24.109784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2ebf-4427-4a34-a347-6e3b7239b03a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:24.110090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2ebf-4427-4a34-a347-6e3b7239b03a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:24.110437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2ebf-4427-4a34-a347-6e3b7239b03a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:24.112365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 809/1633] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 9 07:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:24.120597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0562718-c6b3-4651-8edb-859a96fbb0d4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:24.120957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0562718-c6b3-4651-8edb-859a96fbb0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:24.121615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0562718-c6b3-4651-8edb-859a96fbb0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:24.168292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0562718-c6b3-4651-8edb-859a96fbb0d4 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:24.169548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0562718-c6b3-4651-8edb-859a96fbb0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:24.193386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 825/1634] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 9 07:02:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:24.199009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325120c-6c73-49c4-a5b7-ba82022c73a5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:24.199294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325120c-6c73-49c4-a5b7-ba82022c73a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:24.199572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325120c-6c73-49c4-a5b7-ba82022c73a5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:24.201206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 810/1635] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 9 07:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:24.206710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ec812-09db-4991-b442-381306a6b48c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:24.206993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ec812-09db-4991-b442-381306a6b48c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:24.207277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ec812-09db-4991-b442-381306a6b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:24.245436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bc3ec812-09db-4991-b442-381306a6b48c None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:24.246711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc3ec812-09db-4991-b442-381306a6b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:24.272836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 826/1636] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:02:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:24.360109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325120c-6c73-49c4-a5b7-ba82022c73a5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:24.377239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325120c-6c73-49c4-a5b7-ba82022c73a5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:24.383142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:24.383945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:24.384237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:24.384529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:24.398086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:24.400014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:24.420733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 811/1637] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:02:30.187479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:30.188077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:30.188151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:30.249629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] MFA Rules not processed for user `cc885026afe041cca086367bb67d8a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:30.250880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:30.365847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:30.393473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 827/1638] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:30.412237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:30.426251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1afd3f-bd23-4625-b358-456230b586dc service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:30.447008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c77ff1-14cb-4a3b-aabb-96e4409469e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:30.448345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77ff1-14cb-4a3b-aabb-96e4409469e1 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:30.448865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77ff1-14cb-4a3b-aabb-96e4409469e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:30.449318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77ff1-14cb-4a3b-aabb-96e4409469e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:30.513613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77ff1-14cb-4a3b-aabb-96e4409469e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:30.522582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77ff1-14cb-4a3b-aabb-96e4409469e1 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:30.574793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 812/1639] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:34.597010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:34.794512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0f393-26a5-408e-9234-760c227eb310 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:34.817978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:34.819380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:34.819707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:34.820043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] PATH_INFO: `/v3/users/4fa7298e183f45e48e02419abecda282` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:34.821297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa7298e183f45e48e02419abecda282)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:34.823936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:34.928430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa7298e183f45e48e02419abecda282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:34.940328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa7298e183f45e48e02419abecda282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:34.951665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa7298e183f45e48e02419abecda282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:34.952618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa7298e183f45e48e02419abecda282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:34.986601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa7298e183f45e48e02419abecda282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:35.004146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 828/1640] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:34 2022] DELETE /identity/v3/users/4fa7298e183f45e48e02419abecda282 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:35.005672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:35.006512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:35.007057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:35.069438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:35.071047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:35.124830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:35.282703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 813/1641] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:35.345195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:35.348837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:35.512051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926657e-f96f-4c18-a661-10583349d268 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:35.516861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:35.517497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:35.517781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:35.518051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:35.543419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:35.544952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:35.566759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 829/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:36.256092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:36.257620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:36.284517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:36.291041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:36.293037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fdef6-5e66-428f-9c12-782c519ddcb2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:36.308754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:36.311437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:36.311752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:36.312087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] PATH_INFO: `/v3/users/2b4d2df023624527936d713ad7ab556c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:36.314008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2553b-ce5d-4b35-aabd-7617f883e0b8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:36.314931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4d2df023624527936d713ad7ab556c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:36.317317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 07:02:36.317317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) Aug 09 07:02:36.317317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: Aug 09 07:02:36.330498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:36.336379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:36.342874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:36.343155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:36.343432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] PATH_INFO: `/v3/users/edf8e24e147149f1b05fb34cb55d154b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:36.347113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf8e24e147149f1b05fb34cb55d154b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:36.363722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:36.410868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:36.537463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf8e24e147149f1b05fb34cb55d154b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.558755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf8e24e147149f1b05fb34cb55d154b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.564413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4d2df023624527936d713ad7ab556c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.579858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf8e24e147149f1b05fb34cb55d154b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.581947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf8e24e147149f1b05fb34cb55d154b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.606484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4d2df023624527936d713ad7ab556c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.608253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4d2df023624527936d713ad7ab556c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.620573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ae06770f-6f3a-4417-a2ac-5f70adbbce0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf8e24e147149f1b05fb34cb55d154b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.628634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4d2df023624527936d713ad7ab556c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.631591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 830/1643] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:36 2022] DELETE /identity/v3/users/edf8e24e147149f1b05fb34cb55d154b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:36.637238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469937-f125-44bc-b719-cff32d03d82c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:36.637527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469937-f125-44bc-b719-cff32d03d82c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:36.637804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469937-f125-44bc-b719-cff32d03d82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:36.683079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bc949f3-5d3f-448f-a97b-3d39a9be4b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4d2df023624527936d713ad7ab556c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:36.703673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 814/1644] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:36 2022] DELETE /identity/v3/users/2b4d2df023624527936d713ad7ab556c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:36.703673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7932b-ac06-4602-8545-6224b1e806fd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:36.703673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7932b-ac06-4602-8545-6224b1e806fd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:36.705146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7932b-ac06-4602-8545-6224b1e806fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:36.714489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2d469937-f125-44bc-b719-cff32d03d82c None None] MFA Rules not processed for user `20ef078fe46d418c95f35e1f0bdd2f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:36.719232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2d469937-f125-44bc-b719-cff32d03d82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:36.750506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-efc7932b-ac06-4602-8545-6224b1e806fd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:36.751799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-efc7932b-ac06-4602-8545-6224b1e806fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:36.905342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 831/1645] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:36.907067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:36.907285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:36.908409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:36.942769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 815/1646] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:36.950498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:36.950797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:36.951077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:36.957784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:36.959774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:36.982986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 832/1647] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:36.993169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] MFA Rules not processed for user `20ef078fe46d418c95f35e1f0bdd2f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:36.998697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:37.031946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 816/1648] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:37.033889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:37.035701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:37.049816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:37.053705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:37.188929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:37.300469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d5270-efd4-4cd8-bcbe-0975a8579c7f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:37.324389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:37.325051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:37.325335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:37.326206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:37.337715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1556ec4-fcfa-4f93-8512-9a02a2eba338 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:37.356698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:37.356887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:37.357805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:37.358092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:37.358456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:37.361431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:37.387537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:37.389652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:37.396470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 833/1649] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:37.396939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:37.414396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:37.418980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 817/1650] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:37.517535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822b26c-95c8-41c6-ba88-72d8f1a1c2c5 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:37.532401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff36eb2-5d4d-4f1e-8e10-4b86df224e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:37.533605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36eb2-5d4d-4f1e-8e10-4b86df224e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:37.535072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36eb2-5d4d-4f1e-8e10-4b86df224e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:37.535498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36eb2-5d4d-4f1e-8e10-4b86df224e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:37.557897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:37.559454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:37.568288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff36eb2-5d4d-4f1e-8e10-4b86df224e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:37.573825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff36eb2-5d4d-4f1e-8e10-4b86df224e93 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:37.591547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed03bc6-cbb5-44cd-8687-6413ff7611f8 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:37.606805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:37.618178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:37.618893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:37.619180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:37.619643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] PATH_INFO: `/v3/projects/a20ca03aff6048049ace8e8bcafb6ee1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:37.622043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a20ca03aff6048049ace8e8bcafb6ee1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:37.627056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 834/1651] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:37.628254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:37.801281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a20ca03aff6048049ace8e8bcafb6ee1', 'request_id': 'req-18d0f673-3538-4612-8df8-a81cbc1c9246'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:37.814103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a20ca03aff6048049ace8e8bcafb6ee1', 'request_id': 'req-18d0f673-3538-4612-8df8-a81cbc1c9246'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:37.825486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a20ca03aff6048049ace8e8bcafb6ee1', 'request_id': 'req-18d0f673-3538-4612-8df8-a81cbc1c9246'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:37.852799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] The token cache is being invalidate because project a20ca03aff6048049ace8e8bcafb6ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:37.853153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:37.859272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-18d0f673-3538-4612-8df8-a81cbc1c9246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:37.861230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 818/1652] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:37 2022] DELETE /identity/v3/projects/a20ca03aff6048049ace8e8bcafb6ee1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:37.866351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:37.866779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:37.867180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:37.907650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:37.911328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:38.049203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 835/1653] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:38.057837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.058120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.058445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.089293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:38.090553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:38.102037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 819/1654] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:38.104149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.114790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6486517-b69e-43c7-a442-aae154c5ab8c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.120826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.121473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.121754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.122028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.122829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:38.123145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.124821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:38.144530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.155643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 836/1655] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:38.155998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a46978-ccb0-4310-aca3-f926761af5c5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.165384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.166875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.167293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.167699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.168479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.171322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:38.211184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 820/1656] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:38 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:38.217616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.231644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f39f8-629c-41ae-af56-0adc1ca8bdc1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.324554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.325336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.325874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.326350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.327202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.330045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:38.372292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.373418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 837/1657] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:38 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:38.386590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418560a8-2c95-4511-a37c-07b342df2d6f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.401607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.403512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.403936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.404318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.405146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.407429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:38.427123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.436782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 821/1658] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:38.441486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6892b-7128-463e-861b-cee33637bec0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.450742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096d45d-fa2f-47d1-9c30-3b44edb35f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.451716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096d45d-fa2f-47d1-9c30-3b44edb35f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.452298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096d45d-fa2f-47d1-9c30-3b44edb35f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.452775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096d45d-fa2f-47d1-9c30-3b44edb35f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.453702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096d45d-fa2f-47d1-9c30-3b44edb35f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.455951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096d45d-fa2f-47d1-9c30-3b44edb35f10 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:38.474335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.475135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 838/1659] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:38.487768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa73b1-50c5-48c6-a1f4-306df9b28d1d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.493591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95003c7-b589-4d03-85a3-6f3aac33c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.495035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95003c7-b589-4d03-85a3-6f3aac33c60e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.496103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95003c7-b589-4d03-85a3-6f3aac33c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.497217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95003c7-b589-4d03-85a3-6f3aac33c60e admin admin] PATH_INFO: `/v3/projects/3917698c574d4f52a6cb662c58b255e5/users/b3ff924573774bb5b9f318787597bd83/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.524247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95003c7-b589-4d03-85a3-6f3aac33c60e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3917698c574d4f52a6cb662c58b255e5, user_id=b3ff924573774bb5b9f318787597bd83, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.527565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95003c7-b589-4d03-85a3-6f3aac33c60e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:38.562549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.562999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.563457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.567652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 822/1660] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:38 2022] PUT /identity/v3/projects/3917698c574d4f52a6cb662c58b255e5/users/b3ff924573774bb5b9f318787597bd83/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:38.604023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] MFA Rules not processed for user `b3ff924573774bb5b9f318787597bd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:38.605447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:38.710312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd92e0-f4c7-4a5b-83a7-7147e143e011 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.710819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd92e0-f4c7-4a5b-83a7-7147e143e011 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.711359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd92e0-f4c7-4a5b-83a7-7147e143e011 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.713862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 823/1661] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 9 07:02:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 07:02:38.719202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:38.721431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af636794-b7e1-4a4e-8b97-2d52166cb322 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.721919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af636794-b7e1-4a4e-8b97-2d52166cb322 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.722831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af636794-b7e1-4a4e-8b97-2d52166cb322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.739022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 839/1662] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:38.742652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:38.840169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-af636794-b7e1-4a4e-8b97-2d52166cb322 None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:38.849157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af636794-b7e1-4a4e-8b97-2d52166cb322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:38.897301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:38.959301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70579f-58db-4519-a038-b2131119d1c2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:38.979468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:38.980631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:38.981029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:38.981947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:38.983287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:38.985680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:39.010397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 824/1663] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:39.011984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.012467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.012980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.054339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 840/1664] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:38 2022] POST /identity/v3/projects => generated 365 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:39.059154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.061981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.101702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] MFA Rules not processed for user `4e528f53a4b94e0fab0d90e0dbe07216`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:39.106747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.171282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aa6bc-9ff1-4b44-af9a-71e0d09168ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:39.175514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:39.176217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.177558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.178378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.190531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:39.192529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:39.206598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 841/1665] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:39.207464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.245391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 825/1666] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:39.246730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.248490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.354038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20bce-78b4-4340-a7b3-f6a8d8587409 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:39.364741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:39.365386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.365665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.365945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.366625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:39.368596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:39.379999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c039988-ab8c-4fcc-92e9-d5e1022b275d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:39.387740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:39.388421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.388781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.389428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] PATH_INFO: `/v3/projects/9253508f842f4eb28b9971b1b1a47572` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.392578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9253508f842f4eb28b9971b1b1a47572)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:39.395799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:39.427737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 842/1667] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:39 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:39.428664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.430671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.448254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebc685-777a-43bd-b388-7c21623ba217 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:39.476731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:39.478021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.479415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.480067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] PATH_INFO: `/v3/projects/9ba36b111393480a9ef36b94ec094614` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.481764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba36b111393480a9ef36b94ec094614)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:39.483175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 07:02:39.483175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 07:02:39.483175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: Aug 09 07:02:39.486873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:39.502197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9253508f842f4eb28b9971b1b1a47572', 'request_id': 'req-29eb43b5-0709-4d7d-a046-c61afa80848b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.517952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9253508f842f4eb28b9971b1b1a47572', 'request_id': 'req-29eb43b5-0709-4d7d-a046-c61afa80848b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.523761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:39.524766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:39.547183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9253508f842f4eb28b9971b1b1a47572', 'request_id': 'req-29eb43b5-0709-4d7d-a046-c61afa80848b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.579707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] The token cache is being invalidate because project 9253508f842f4eb28b9971b1b1a47572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:39.581619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.583809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.588608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 826/1668] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:39 2022] DELETE /identity/v3/projects/9253508f842f4eb28b9971b1b1a47572 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:39.589932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.594457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.642177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba36b111393480a9ef36b94ec094614', 'request_id': 'req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.655568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba36b111393480a9ef36b94ec094614', 'request_id': 'req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.677154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba36b111393480a9ef36b94ec094614', 'request_id': 'req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.704559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] The token cache is being invalidate because project 9ba36b111393480a9ef36b94ec094614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:39.705900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.707179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:39.710338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 843/1669] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:39 2022] DELETE /identity/v3/projects/9ba36b111393480a9ef36b94ec094614 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:39.716875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.716875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.793354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb43b5-0709-4d7d-a046-c61afa80848b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:39.891474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d93f12-31a1-441b-8c89-0bbdcbe0e84a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:39.907498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:39.908243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.908626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.909006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.909773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:39.912566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:39.939910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 844/1670] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:39.940940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:39.944700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:39.980405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:39.984200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:39.986779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:39.987195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:39.991960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.000828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2482d-a728-4066-9278-1fb03570d344 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.014149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.015767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.016788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.017884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] PATH_INFO: `/v3/users/e3e88d807ab946f59436a46ce32d7284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.021293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e88d807ab946f59436a46ce32d7284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:40.031037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:40.115018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:40.116871 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4475165-bd23-4082-bdb1-ce6bdfaca2c0 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:40.130438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 827/1671] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:40.132067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d11d73-1e7c-4e66-8112-7719a743e784 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.132862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d11d73-1e7c-4e66-8112-7719a743e784 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.133248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d11d73-1e7c-4e66-8112-7719a743e784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.161480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-64d11d73-1e7c-4e66-8112-7719a743e784 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:40.162919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-64d11d73-1e7c-4e66-8112-7719a743e784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.171182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e88d807ab946f59436a46ce32d7284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:40.194606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e88d807ab946f59436a46ce32d7284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:40.213823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 828/1672] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:40.219226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa84f1-079e-47b8-82f8-eb72058a217c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.220502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa84f1-079e-47b8-82f8-eb72058a217c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.221422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa84f1-079e-47b8-82f8-eb72058a217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.231695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e88d807ab946f59436a46ce32d7284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:40.234459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e88d807ab946f59436a46ce32d7284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:40.271356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e88d807ab946f59436a46ce32d7284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:40.297706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 845/1673] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:39 2022] DELETE /identity/v3/users/e3e88d807ab946f59436a46ce32d7284 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:40.297706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:40.298793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.299686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-06fa84f1-079e-47b8-82f8-eb72058a217c None None] MFA Rules not processed for user `cc885026afe041cca086367bb67d8a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:40.313539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:40.315182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06fa84f1-079e-47b8-82f8-eb72058a217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.342345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:40.496031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a19e4-0d11-4a75-83b8-edf0cc022d25 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.502042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 829/1674] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:40.503405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374918e-9c1d-4899-8fdc-9090e68fd727 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.503958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374918e-9c1d-4899-8fdc-9090e68fd727 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.504649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374918e-9c1d-4899-8fdc-9090e68fd727 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.505728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 830/1675] 10.222.0.30 () {54 vars in 961 bytes} [Tue Aug 9 07:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:40.507025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.507133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.507385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.523632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857ca0f-7c7a-4aaa-aa8f-e77e2bb513fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.524497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857ca0f-7c7a-4aaa-aa8f-e77e2bb513fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.525556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857ca0f-7c7a-4aaa-aa8f-e77e2bb513fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.526422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857ca0f-7c7a-4aaa-aa8f-e77e2bb513fc admin admin] PATH_INFO: `/v3/projects/bdfa2917fd6b4b639c9b86e4714529c5/users/ac66169770474696a9bfcd1963e1f209/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.555521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857ca0f-7c7a-4aaa-aa8f-e77e2bb513fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfa2917fd6b4b639c9b86e4714529c5, user_id=ac66169770474696a9bfcd1963e1f209, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:40.559475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857ca0f-7c7a-4aaa-aa8f-e77e2bb513fc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:40.592347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:40.596593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.597640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 846/1676] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:40 2022] PUT /identity/v3/projects/bdfa2917fd6b4b639c9b86e4714529c5/users/ac66169770474696a9bfcd1963e1f209/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:40.603315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.604346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.606142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.667450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:40.669393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.779215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 847/1677] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Aug 9 07:02:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:40.791155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:40.812103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 831/1678] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:40.815366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:40.832990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ca47a-986f-4f45-8922-bd35b54e9de5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.851661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.854015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.854325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.854696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.855554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:40.858105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:40.872015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6b72b-2c07-48eb-b81e-dde36ee2ef0d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.877761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 848/1679] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:40.879024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:40.884343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.886271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.886716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.887980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.891339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2106-5dd7-4ef4-bcf4-8b4d1c22fba3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.898320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:40.909045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.912676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.916617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.917085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.917922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:40.919817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:40.927213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:40.934991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:40.940137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:40.953556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 849/1680] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:40.955037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:40.960118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 832/1681] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:40.965777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:40.979126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e996aa-15f8-4c63-8367-7fe0e211a423 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.985718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69378335-1b9d-49e7-a280-28d6573b83f0 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:40.989617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ff33-d741-485f-b5aa-f21139a43baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.991494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ff33-d741-485f-b5aa-f21139a43baa admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.992983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ff33-d741-485f-b5aa-f21139a43baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:40.994081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ff33-d741-485f-b5aa-f21139a43baa admin admin] PATH_INFO: `/v3/projects/bdfa2917fd6b4b639c9b86e4714529c5/users/ac66169770474696a9bfcd1963e1f209/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:40.996801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:40.999132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:40.999462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ff33-d741-485f-b5aa-f21139a43baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfa2917fd6b4b639c9b86e4714529c5, user_id=ac66169770474696a9bfcd1963e1f209, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.002299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ff33-d741-485f-b5aa-f21139a43baa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.002755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.003145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.004623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.006861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.038022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 850/1682] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:40 2022] PUT /identity/v3/projects/bdfa2917fd6b4b639c9b86e4714529c5/users/ac66169770474696a9bfcd1963e1f209/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:41.054686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.055027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.055355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.100938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 833/1683] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:40 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:41.110165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:41.167491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] MFA Rules not processed for user `ac66169770474696a9bfcd1963e1f209`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:41.168770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:41.308043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:41.339990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a79156-85c2-4e81-a2cf-024f749b66f4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:41.357091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:41.357742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.358046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.358304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.374606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.376663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.420607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 851/1684] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:41.428795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:41.430486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:41.455511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 834/1685] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:41 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:41.459586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:41.483008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914383b1-e599-4f1c-ae0e-579561d25166 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:41.496990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:41.497654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.497993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.498205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.501525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.503293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.552550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 835/1686] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:02:41.553411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:41.572792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:41.588897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e81ce-00c7-4e91-8590-154540588194 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:41.616646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:41.619571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.629740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.630049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.630757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.637245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd478-4f0b-41dd-9b22-7cb06b232dc2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:41.638171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.647937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578c067-23cd-4311-bf05-186fcb866527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:41.648613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578c067-23cd-4311-bf05-186fcb866527 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.648896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578c067-23cd-4311-bf05-186fcb866527 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.649183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578c067-23cd-4311-bf05-186fcb866527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.670870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 836/1687] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:41.676830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:41.685366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578c067-23cd-4311-bf05-186fcb866527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.686823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578c067-23cd-4311-bf05-186fcb866527 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.707255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d4d9-4009-4cde-b8d1-68914dadb591 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:41.713964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 852/1688] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:41.744547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.744884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.745173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.746024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493731f4-0003-4fa0-931a-7ce74657dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:41.747290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493731f4-0003-4fa0-931a-7ce74657dc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.747290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493731f4-0003-4fa0-931a-7ce74657dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.747516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493731f4-0003-4fa0-931a-7ce74657dc1d admin admin] PATH_INFO: `/v3/projects/c2648d1bb6e94dbe8b5a7a877de3080c/users/5f8894118c8b4c0bb412e5a8d05f020f/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.802260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493731f4-0003-4fa0-931a-7ce74657dc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2648d1bb6e94dbe8b5a7a877de3080c, user_id=5f8894118c8b4c0bb412e5a8d05f020f, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:41.805173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493731f4-0003-4fa0-931a-7ce74657dc1d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:41.824014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] MFA Rules not processed for user `19da3c426c6e4c4a889ab5017804c92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:41.831736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:41.839410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 837/1689] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:41 2022] PUT /identity/v3/projects/c2648d1bb6e94dbe8b5a7a877de3080c/users/5f8894118c8b4c0bb412e5a8d05f020f/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:41.860035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcddf1-0044-433f-8de7-e9bd64d56c0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:41.883422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcddf1-0044-433f-8de7-e9bd64d56c0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:41.883863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcddf1-0044-433f-8de7-e9bd64d56c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:41.948657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-98bcddf1-0044-433f-8de7-e9bd64d56c0d None None] MFA Rules not processed for user `0bb72c5f52fc498aa60eecba3bd5248a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:41.950723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-98bcddf1-0044-433f-8de7-e9bd64d56c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:42.156148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 838/1690] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:42.157446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:42.158261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:42.158261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:42.170463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 853/1691] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:42.174778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:42.261971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] MFA Rules not processed for user `5f8894118c8b4c0bb412e5a8d05f020f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:42.263223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:42.356913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d650eb-dc83-44df-aaf4-0652ed80e5fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:42.364206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:42.367533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:42.368472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:42.370539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:42.374130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:42.428826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:42.434136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 839/1692] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:42.434136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:42.436641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:42.544285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:42.545832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f110d8-b63a-43be-aa35-d2b889baf6d3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:42.566598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:42.568709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 854/1693] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:42.574777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:42.574966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:42.575113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:42.605189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692f9dc-45fb-43b1-941b-bb22c82f85f9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:42.615000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:42.615715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:42.615998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:42.616272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] PATH_INFO: `/v3/projects/ccfdb0bba51840ae88e3569f31187be7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:42.619988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfdb0bba51840ae88e3569f31187be7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:42.621485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] MFA Rules not processed for user `19da3c426c6e4c4a889ab5017804c92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:42.623962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:42.625124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:42.642542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 855/1694] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:42.644785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:42.647892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:42.772615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfdb0bba51840ae88e3569f31187be7', 'request_id': 'req-68c0e162-becf-42a2-8317-72e051f12be3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:42.795008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfdb0bba51840ae88e3569f31187be7', 'request_id': 'req-68c0e162-becf-42a2-8317-72e051f12be3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:42.820677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfdb0bba51840ae88e3569f31187be7', 'request_id': 'req-68c0e162-becf-42a2-8317-72e051f12be3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:42.896533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] The token cache is being invalidate because project ccfdb0bba51840ae88e3569f31187be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:42.896840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:42.904035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-68c0e162-becf-42a2-8317-72e051f12be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:42.906699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 840/1695] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:42 2022] DELETE /identity/v3/projects/ccfdb0bba51840ae88e3569f31187be7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:42.915957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:42.917146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:42.917438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:42.956355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a137777-995a-4c1e-8fda-d96425841b7a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:43.005515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:43.007302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:43.167779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 841/1696] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:43.167779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:43.168188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:43.168188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.168188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.168188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.170695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:43.171068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:43.214919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adfa68-eebf-414b-8778-bd1c2ef1b756 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:43.227766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:43.231464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.232794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.233082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.237543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:43.256437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:43.287838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:43.289734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccbb9c-afc6-42fb-bb82-22ce52196a6b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:43.311544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 856/1697] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:43.313329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.313758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.314187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.376284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:43.377551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:43.407307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 857/1698] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:43.411251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:43.423429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:43.430301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945c15-3ce5-4217-affc-d285ceff80e8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:43.430807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:43.452154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:43.454540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.455565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.455964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.457360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:43.458060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:43.463652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:43.466061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 842/1699] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:43.486595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 858/1700] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:02:43.490799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:43.503538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:43.510002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1eb689-a930-4d96-bbed-1654e7630dff service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:43.521486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:43.523716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.524140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.524570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.527146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:43.530439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:43.534284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4027-dcdf-43b9-aa74-f0cb0d3a3618 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:43.552334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:43.554746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.555990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.557066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.562229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:43.575352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:43.599398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 843/1701] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:43 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:43.604284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:43.777018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:43.783626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:43.842275 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7085818-6c0f-44e1-9c26-6cfca47d708d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:43.842840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 859/1702] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:43.863638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:43.864337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:43.864624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:43.864899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:43.865532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:43.876839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:43.985271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 844/1703] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:43 2022] POST /identity/v3/users => generated 283 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:43.989292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:44.032895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eae0a-7d8c-4f21-83d4-49c612a6944d service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:44.044335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:44.044981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:44.045286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:44.045567 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:44.046260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:44.049097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:44.080249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 860/1704] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:02:44.081504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:44.104939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a44d6-457f-4698-b080-74fa8098ebb1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:44.137285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb316674-4e2e-480c-8d1f-5ba19acd332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:44.138662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb316674-4e2e-480c-8d1f-5ba19acd332d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:44.143922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb316674-4e2e-480c-8d1f-5ba19acd332d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:44.145649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb316674-4e2e-480c-8d1f-5ba19acd332d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:44.149061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb316674-4e2e-480c-8d1f-5ba19acd332d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:44.154167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb316674-4e2e-480c-8d1f-5ba19acd332d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:44.213673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:44.214206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 845/1705] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:44.233086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a276d6-8e43-4c69-9beb-409c517926e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:44.263872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ace001-987f-4d3d-bf87-7084aa86945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:44.264648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ace001-987f-4d3d-bf87-7084aa86945b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:44.265003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ace001-987f-4d3d-bf87-7084aa86945b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:44.265319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ace001-987f-4d3d-bf87-7084aa86945b admin admin] PATH_INFO: `/v3/projects/4dc45f5fc6e0420abe2e4310dc4a47cf/users/964ca732fc884654bd10f7b51b64a9aa/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:44.334001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ace001-987f-4d3d-bf87-7084aa86945b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc45f5fc6e0420abe2e4310dc4a47cf, user_id=964ca732fc884654bd10f7b51b64a9aa, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:44.336785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ace001-987f-4d3d-bf87-7084aa86945b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:44.399649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 861/1706] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:44 2022] PUT /identity/v3/projects/4dc45f5fc6e0420abe2e4310dc4a47cf/users/964ca732fc884654bd10f7b51b64a9aa/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:44.412026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:44.412427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:44.412735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:44.514299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] MFA Rules not processed for user `964ca732fc884654bd10f7b51b64a9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:44.514299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:44.520114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:44.520667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:44.521637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:44.690191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:44.694731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:44.744872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 846/1707] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:44.747207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:44.963119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c9eec-4ff6-468f-883c-73d5ca018348 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:44.996166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:44.996806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:45.002044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:45.002320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:45.005999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:45.014751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:45.067467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:45.086018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 862/1708] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:02:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 567 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:02:45.174835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:45.177288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 847/1709] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:44 2022] POST /identity/v3/projects => generated 363 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:45.189506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:45.234606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4507de3-518e-4763-a462-44e3efeea9ae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:45.345618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:45.348011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:45.459303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:45.460279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:45.460727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:45.461142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:45.463188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:45.468833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:45.526336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 863/1710] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:45 2022] POST /identity/v3/users => generated 288 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:45.529934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:45.570254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28fac-72e1-40e0-951c-276ecc929b75 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:45.585151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:45.585987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:45.586330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:45.586621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:45.590529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:45.595955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:45.626052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b085a-fee6-411a-8f6f-de7ec7307506 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:45.627316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 864/1711] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:45.628421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:45.632839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:45.660392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:45.661135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:45.661422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:45.661698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] PATH_INFO: `/v3/users/5f8894118c8b4c0bb412e5a8d05f020f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:45.665015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8894118c8b4c0bb412e5a8d05f020f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:45.676235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:45.860925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be8d8-1592-49f9-ad41-5b6a19470d5f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:45.870067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:45.870906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:45.871296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:45.871645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:45.921015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8894118c8b4c0bb412e5a8d05f020f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:45.945613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8894118c8b4c0bb412e5a8d05f020f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:45.950666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8894118c8b4c0bb412e5a8d05f020f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:45.970194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8894118c8b4c0bb412e5a8d05f020f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:46.005979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d038b536-2020-4303-914c-f66bf98b00b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8894118c8b4c0bb412e5a8d05f020f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:46.012184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:46.014171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:46.027363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 865/1712] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:02:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:02:46.032476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 848/1713] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:45 2022] DELETE /identity/v3/users/5f8894118c8b4c0bb412e5a8d05f020f => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:02:46.033764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:46.035137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:46.035431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:46.036718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:46.044092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:46.061616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de6335-4e25-4dac-aaf6-06a02c386b9f service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:46.087041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:46.090403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:46.254635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:46.255372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:46.255676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:46.256492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] PATH_INFO: `/v3/projects/53820fd684884f64801da679533a93dd/users/3ee34c719db04b9fbcdff7a9429cac26/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:46.277443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 849/1714] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:46.287798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53820fd684884f64801da679533a93dd, user_id=3ee34c719db04b9fbcdff7a9429cac26, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:46.291302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:46.318341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 866/1715] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:46 2022] PUT /identity/v3/projects/53820fd684884f64801da679533a93dd/users/3ee34c719db04b9fbcdff7a9429cac26/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:46.324394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:46.327094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:46.345376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:46.347810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:46.362885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8867894-327b-4484-b846-0d6661e7b1a1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:46.388027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:46.570205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709bf51-2ad7-4def-abd9-0ea960be6a07 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:46.588571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:46.589217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:46.589502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:46.589781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:46.609304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:46.610020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:46.616592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:46.617960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:46.621079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:46.622756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:46.681844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 850/1716] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:46.685968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:46.734843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff117fa-2d5a-4474-8c05-5e6db52ca424 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:46.769689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24f68d-b9e4-4935-bb89-a032d25094dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:46.774100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f68d-b9e4-4935-bb89-a032d25094dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:46.774592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f68d-b9e4-4935-bb89-a032d25094dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:46.782678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f68d-b9e4-4935-bb89-a032d25094dc admin admin] PATH_INFO: `/v3/projects/53820fd684884f64801da679533a93dd/users/3ee34c719db04b9fbcdff7a9429cac26/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:46.793518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24f68d-b9e4-4935-bb89-a032d25094dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=53820fd684884f64801da679533a93dd, user_id=3ee34c719db04b9fbcdff7a9429cac26, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:46.809697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24f68d-b9e4-4935-bb89-a032d25094dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:46.904264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 851/1717] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:46 2022] PUT /identity/v3/projects/53820fd684884f64801da679533a93dd/users/3ee34c719db04b9fbcdff7a9429cac26/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:02:46.915062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:46.916012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:46.916400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:46.969917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:46.972412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:46.989504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 867/1718] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:47.041878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] MFA Rules not processed for user `3ee34c719db04b9fbcdff7a9429cac26`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:47.043217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:47.282973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 852/1719] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:47.297280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:47.299015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:47.407310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:47.444193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14638d-77cc-4421-9b5b-108f57de17cd service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:47.619923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d8793-282b-4357-876f-bad1ce172bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:47.620663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8793-282b-4357-876f-bad1ce172bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:47.620941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8793-282b-4357-876f-bad1ce172bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:47.621213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8793-282b-4357-876f-bad1ce172bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:47.659899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d8793-282b-4357-876f-bad1ce172bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:47.663195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d8793-282b-4357-876f-bad1ce172bb9 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:47.734972 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 868/1720] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:47.942534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:47.969340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527f424-94cc-412a-b58e-8a00a6541336 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:47.986626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:47.987528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:47.988228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:47.988810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:47.996351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:48.227130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:48.228938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:48.269854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14737eb7-75ad-441f-b62f-691c8b2f19b2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:48.275891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14737eb7-75ad-441f-b62f-691c8b2f19b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:48.276869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14737eb7-75ad-441f-b62f-691c8b2f19b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:48.277058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 853/1721] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:48.389990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-14737eb7-75ad-441f-b62f-691c8b2f19b2 None None] MFA Rules not processed for user `19da3c426c6e4c4a889ab5017804c92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:48.394711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-14737eb7-75ad-441f-b62f-691c8b2f19b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:48.783511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:48.785308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:48.790153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 869/1722] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:02:48.835213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:49.008815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815b765-c983-4aad-9678-9cf8bc54f1f7 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:49.017458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:49.018579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:49.018909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:49.019191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] PATH_INFO: `/v3/projects/c2648d1bb6e94dbe8b5a7a877de3080c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:49.022515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2648d1bb6e94dbe8b5a7a877de3080c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:49.035534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:49.237410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2648d1bb6e94dbe8b5a7a877de3080c', 'request_id': 'req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:49.258458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2648d1bb6e94dbe8b5a7a877de3080c', 'request_id': 'req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:49.282464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2648d1bb6e94dbe8b5a7a877de3080c', 'request_id': 'req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:49.321343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] The token cache is being invalidate because project c2648d1bb6e94dbe8b5a7a877de3080c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:49.323289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:49.325455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-69bfda03-c5ce-49e2-a227-b1a72e1f38f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:49.334614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 854/1723] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:48 2022] DELETE /identity/v3/projects/c2648d1bb6e94dbe8b5a7a877de3080c => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:02:49.340663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:49.341381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:49.342515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:49.417478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:49.421158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:49.611761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 870/1724] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:49.617464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:49.617748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:49.618028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:49.674108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:49.675332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:49.701700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 855/1725] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:49.703017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:49.718582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931624-5dff-430d-a975-cea3dde8e615 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:49.732293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:49.732941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:49.733227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:49.733503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:49.734171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:49.734481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:49.736162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:49.795590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 871/1726] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:02:49.799736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:49.813336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e2730-d329-4a6f-9f18-adde39a7548a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:49.833975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:49.834283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:49.834568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:49.834844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:49.835486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:49.837712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:49.913164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 856/1727] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:49 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:02:49.927717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:49.941368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4ab4-49fb-47eb-afa5-3c08cf70fae9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:49.999285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:50.078166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1744c-8beb-461a-88b2-92d2beb4ac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:50.079136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1744c-8beb-461a-88b2-92d2beb4ac2f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.079623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1744c-8beb-461a-88b2-92d2beb4ac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.079857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1744c-8beb-461a-88b2-92d2beb4ac2f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.080766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1744c-8beb-461a-88b2-92d2beb4ac2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:50.083638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1744c-8beb-461a-88b2-92d2beb4ac2f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:50.115807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 872/1728] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:49 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:50.121411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:50.154184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b541cee-3007-4b30-a1f0-d5f807de324f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:50.166739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:50.167700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.169293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.170306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.172574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:50.183811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:50.186123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf92928-e202-4b08-bf91-bbf8e61b55f6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.186123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf92928-e202-4b08-bf91-bbf8e61b55f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.186306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf92928-e202-4b08-bf91-bbf8e61b55f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.219040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 857/1729] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:50.228409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:50.230992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1cf92928-e202-4b08-bf91-bbf8e61b55f6 None None] MFA Rules not processed for user `4e528f53a4b94e0fab0d90e0dbe07216`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:50.234384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1cf92928-e202-4b08-bf91-bbf8e61b55f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:50.272024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f8fc3-5784-4e00-992a-529e091d419d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:50.288782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:50.289668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.291727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.292356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.293253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:50.297244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:50.321202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 858/1730] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:50.326325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:50.354036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e75e0-b97b-4346-9f49-13352d75e8d1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:50.361245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:50.362524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.362815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.363968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] PATH_INFO: `/v3/projects/c37d5a3b88ab4b169e3f3e4e423a4498/users/03fe851af95548e6bd74b1942fe581d7/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.383232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37d5a3b88ab4b169e3f3e4e423a4498, user_id=03fe851af95548e6bd74b1942fe581d7, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:50.386250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:50.397844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 873/1731] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:50.434323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 859/1732] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:50 2022] PUT /identity/v3/projects/c37d5a3b88ab4b169e3f3e4e423a4498/users/03fe851af95548e6bd74b1942fe581d7/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:50.436079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.436555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.436653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.466684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] MFA Rules not processed for user `03fe851af95548e6bd74b1942fe581d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:50.468053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:50.567554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 874/1733] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:50.572163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:50.661057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:50.741867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa050395-9cd0-4d85-976d-f836bf8b9945 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:50.758314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:50.763446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.763446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.766229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.766229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:50.772033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:50.826626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 860/1734] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:50 2022] POST /identity/v3/projects => generated 373 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:50.834585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:50.860840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088b439-6f9d-4b70-a80a-c0939e3ab4de None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:50.930295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:50.973168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:50.975934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:50.976244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:50.976517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:50.978942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:50.983185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:51.058858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 875/1735] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:50 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:51.059404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:51.080055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d438-7b93-411c-b838-3068a640e520 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:51.094346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:51.094983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.095269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.095546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.110697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:51.115192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:51.131425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 861/1736] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:51.137529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:51.161131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae26206-eb02-402f-8370-badc83b481ce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:51.180192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:51.181420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.181710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.181983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] PATH_INFO: `/v3/projects/4cae8fad268e4a69acfa71713aaca9a3/users/500bb289055d4f41af0dfb01c121ae9e/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.239803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cae8fad268e4a69acfa71713aaca9a3, user_id=500bb289055d4f41af0dfb01c121ae9e, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:51.245207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:51.270121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 876/1737] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:51 2022] PUT /identity/v3/projects/4cae8fad268e4a69acfa71713aaca9a3/users/500bb289055d4f41af0dfb01c121ae9e/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:51.271184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:51.291515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dc7b6-6d44-42fb-bfe8-9f88444734e4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:51.448922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab39ba-5413-433e-855e-ef07911885fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:51.449685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab39ba-5413-433e-855e-ef07911885fb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.450353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab39ba-5413-433e-855e-ef07911885fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.450683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab39ba-5413-433e-855e-ef07911885fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.451655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab39ba-5413-433e-855e-ef07911885fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:51.456132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab39ba-5413-433e-855e-ef07911885fb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:51.485282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:51.487357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 862/1738] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:02:51.511622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036c06e-f53a-425b-a720-08cd61b73dc8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:51.517014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.517277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.517891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.519640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab7140-eb28-4f1a-8074-bd48f9ca5e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:51.521738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab7140-eb28-4f1a-8074-bd48f9ca5e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.523368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab7140-eb28-4f1a-8074-bd48f9ca5e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.523821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab7140-eb28-4f1a-8074-bd48f9ca5e5b admin admin] PATH_INFO: `/v3/projects/4cae8fad268e4a69acfa71713aaca9a3/users/500bb289055d4f41af0dfb01c121ae9e/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.533437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab7140-eb28-4f1a-8074-bd48f9ca5e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cae8fad268e4a69acfa71713aaca9a3, user_id=500bb289055d4f41af0dfb01c121ae9e, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:51.551392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab7140-eb28-4f1a-8074-bd48f9ca5e5b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:51.572188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:51.579230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:51.582576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 877/1739] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:51 2022] PUT /identity/v3/projects/4cae8fad268e4a69acfa71713aaca9a3/users/500bb289055d4f41af0dfb01c121ae9e/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:02:51.598646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.599105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.599383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.647519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] MFA Rules not processed for user `500bb289055d4f41af0dfb01c121ae9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:51.648952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:51.718439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 863/1740] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Aug 9 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:51.775334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:51.776923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:51.794468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc772a9-59da-4dcb-bcf7-5927f5e053ff None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:51.818309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:51.819878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:51.819878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:51.819878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:51.825986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 878/1741] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:51.833405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:51.855111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:51.856610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:51.865071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:51.939671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 864/1742] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:02:52.106913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:52.109383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:52.135289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19207922-59fd-4a3c-982f-ed992d6ca0b8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:52.150986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:52.156373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:52.156934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:52.157580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:52.194357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:52.197809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:52.240448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 879/1743] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:02:52.288779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:52.408107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4f4c7-505b-4d03-adf6-e5654db67b3e service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:52.427819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:52.429551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:52.430524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:52.432253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] PATH_INFO: `/v3/users/964ca732fc884654bd10f7b51b64a9aa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:52.435260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] RBAC: Authorizing `identity:delete_user(user_id=964ca732fc884654bd10f7b51b64a9aa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:52.440330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:52.520609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:52.523138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:52.551565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554788b0-c467-446d-9c75-9a7772762abf service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:52.568042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:52.569178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:52.570374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:52.572023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:52.604399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964ca732fc884654bd10f7b51b64a9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:52.621948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964ca732fc884654bd10f7b51b64a9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:52.636242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964ca732fc884654bd10f7b51b64a9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:52.654310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964ca732fc884654bd10f7b51b64a9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:52.695366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98d48b43-fb26-4ee7-bc18-939ab6b6301d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964ca732fc884654bd10f7b51b64a9aa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:52.716570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 865/1744] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:52 2022] DELETE /identity/v3/users/964ca732fc884654bd10f7b51b64a9aa => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:52.719175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440e59-8d33-4098-866c-7672c8595ae1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:52.719564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440e59-8d33-4098-866c-7672c8595ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:52.719924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440e59-8d33-4098-866c-7672c8595ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:52.733394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:52.780477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bc440e59-8d33-4098-866c-7672c8595ae1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:52.807025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bc440e59-8d33-4098-866c-7672c8595ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:52.886181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:52.894058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:52.939123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 880/1745] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:53.026254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 866/1746] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:53.216058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:53.218116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:53.255523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624d79f-1763-4ae1-bfd1-c844ea65953c service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:53.462943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0c3ae-ae7b-467c-a7b5-b0213fbe5f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:53.463689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0c3ae-ae7b-467c-a7b5-b0213fbe5f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:53.463984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0c3ae-ae7b-467c-a7b5-b0213fbe5f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:53.464260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0c3ae-ae7b-467c-a7b5-b0213fbe5f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:53.491870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0c3ae-ae7b-467c-a7b5-b0213fbe5f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:53.494255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0c3ae-ae7b-467c-a7b5-b0213fbe5f1c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:53.556730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 881/1747] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:02:53.761469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d0259-89b6-496d-9088-991284e8eecf None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:53.762013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d0259-89b6-496d-9088-991284e8eecf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:53.762291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d0259-89b6-496d-9088-991284e8eecf None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:53.764759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 867/1748] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 9 07:02:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:53.775701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887567d-c399-468c-8bf9-9cd20a923f70 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:53.776043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887567d-c399-468c-8bf9-9cd20a923f70 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:53.776329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887567d-c399-468c-8bf9-9cd20a923f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:53.833902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4887567d-c399-468c-8bf9-9cd20a923f70 None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:53.838037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4887567d-c399-468c-8bf9-9cd20a923f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:53.867004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 882/1749] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:53.996528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d0259-89b6-496d-9088-991284e8eecf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:53.998731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b31d0259-89b6-496d-9088-991284e8eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:54.070739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d0259-89b6-496d-9088-991284e8eecf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:54.084205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfae1e-f768-49aa-bb4f-fb3ce0c51da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:54.084932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfae1e-f768-49aa-bb4f-fb3ce0c51da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:54.085271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfae1e-f768-49aa-bb4f-fb3ce0c51da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:54.086492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfae1e-f768-49aa-bb4f-fb3ce0c51da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:54.133015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfae1e-f768-49aa-bb4f-fb3ce0c51da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:54.141177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfae1e-f768-49aa-bb4f-fb3ce0c51da2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:54.154064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 868/1750] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:02:54.658746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec884b-e94d-48a2-90df-3ae716595783 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:54.659785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec884b-e94d-48a2-90df-3ae716595783 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:54.660281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec884b-e94d-48a2-90df-3ae716595783 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:54.662234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 883/1751] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:02:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:54.843121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:54.843121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:54.843121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:54.936005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:54.953705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:55.103302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:55.103822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:55.104651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:55.192746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] MFA Rules not processed for user `0bb72c5f52fc498aa60eecba3bd5248a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:55.194135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:55.204437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:55.314618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 869/1752] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 09 07:02:55.379718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 884/1753] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:56.420370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:56.423394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:56.459268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8645-5b33-4645-9d5c-a07856df3a11 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:56.480966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:56.485986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:56.487054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:56.494340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] PATH_INFO: `/v3/projects/4dc45f5fc6e0420abe2e4310dc4a47cf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:56.497513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc45f5fc6e0420abe2e4310dc4a47cf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:56.505853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:56.712199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc45f5fc6e0420abe2e4310dc4a47cf', 'request_id': 'req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:56.731346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc45f5fc6e0420abe2e4310dc4a47cf', 'request_id': 'req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:56.778318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc45f5fc6e0420abe2e4310dc4a47cf', 'request_id': 'req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:56.825494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] The token cache is being invalidate because project 4dc45f5fc6e0420abe2e4310dc4a47cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:56.825910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:56.826800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:56.834587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 870/1754] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:56 2022] DELETE /identity/v3/projects/4dc45f5fc6e0420abe2e4310dc4a47cf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:56.835534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:56.859833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b311d-aca1-4d83-a628-eb6b947ff991 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:56.944433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:56.947692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:57.161930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:57.165696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:57.168635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:57.170643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] PATH_INFO: `/v3/users/3ee34c719db04b9fbcdff7a9429cac26` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:57.174677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee34c719db04b9fbcdff7a9429cac26)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:57.189568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:57.244110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:57.358086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2af33d-5d84-402b-9e3f-c4d21d19eb5c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:57.374334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:57.375164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:57.375619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:57.375884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee34c719db04b9fbcdff7a9429cac26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.376205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:57.385631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:57.410968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee34c719db04b9fbcdff7a9429cac26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.417298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee34c719db04b9fbcdff7a9429cac26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.418384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee34c719db04b9fbcdff7a9429cac26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.436681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:57.438329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:57.446638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-230cc1cd-3fcf-4aff-b6cf-95a848d01302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee34c719db04b9fbcdff7a9429cac26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.487908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 871/1756] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:02:57.488402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 885/1756] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:56 2022] DELETE /identity/v3/users/3ee34c719db04b9fbcdff7a9429cac26 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:02:57.490193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:57.496029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:57.552483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320b502-ac6d-4811-baf7-d453fc3fc037 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:57.734411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:57.791757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:57.792558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:57.792946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:57.793328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] PATH_INFO: `/v3/users/b3ff924573774bb5b9f318787597bd83` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:57.794880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ff924573774bb5b9f318787597bd83)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:57.797254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:57.885274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ff924573774bb5b9f318787597bd83'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.892864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ff924573774bb5b9f318787597bd83'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.893588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ff924573774bb5b9f318787597bd83'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.907285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ff924573774bb5b9f318787597bd83'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.930305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ff924573774bb5b9f318787597bd83'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:57.952779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 872/1757] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:57 2022] DELETE /identity/v3/users/b3ff924573774bb5b9f318787597bd83 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:57.954654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:57.955641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:57.956894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:57.997638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:58.002299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:58.153728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 886/1758] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:58.182842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.185094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:58.268783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.270216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:58.291215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e522e-37b8-4c99-abba-3daa6acf7c80 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.291501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a63b58-0bfb-4fb5-a309-e2d5b884cdef None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.296628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3fd1c-1089-4334-ae86-6fd4d0c04ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.296902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.297711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.298147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.298323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3fd1c-1089-4334-ae86-6fd4d0c04ecb service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.298706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3fd1c-1089-4334-ae86-6fd4d0c04ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.298832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] PATH_INFO: `/v3/projects/53820fd684884f64801da679533a93dd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.299155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3fd1c-1089-4334-ae86-6fd4d0c04ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.300874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=53820fd684884f64801da679533a93dd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.302938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.311868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3fd1c-1089-4334-ae86-6fd4d0c04ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.313757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3fd1c-1089-4334-ae86-6fd4d0c04ecb service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.329530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 873/1759] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:02:58.362332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53820fd684884f64801da679533a93dd', 'request_id': 'req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:58.374130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53820fd684884f64801da679533a93dd', 'request_id': 'req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:58.386006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53820fd684884f64801da679533a93dd', 'request_id': 'req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:58.404643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] The token cache is being invalidate because project 53820fd684884f64801da679533a93dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:58.404984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:58.405764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0fb78ced-6270-44a4-b25f-f8ff1ebfb50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:58.412188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 887/1760] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:58 2022] DELETE /identity/v3/projects/53820fd684884f64801da679533a93dd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:58.414577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.415401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.415401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.476469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:58.479493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:58.584017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 874/1761] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:58.595374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.596017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.596485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.620660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:58.622222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:58.632000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 888/1762] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:58.636898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.651819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e159b-8599-437a-bdbe-4c52ed59bcc8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.659437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.660437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.661672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.662162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.663196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:02:58.663786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.666683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.668929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:58.685932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 875/1763] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:02:58.689920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.706673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e895fdb-9706-4a0f-b0ee-417296ba956f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.713623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.714117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.714408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.714681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.715329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.718864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.723830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:58.771863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 889/1764] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:58 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:02:58.773384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.781971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.784209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:58.786322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fbd46-5e10-404d-9921-bd4b5b983bb7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.888953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39de0e0-bf7b-4653-ad88-6ca949c4c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.889583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de0e0-bf7b-4653-ad88-6ca949c4c02b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.889862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de0e0-bf7b-4653-ad88-6ca949c4c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.890136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de0e0-bf7b-4653-ad88-6ca949c4c02b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.890777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39de0e0-bf7b-4653-ad88-6ca949c4c02b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.896395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39de0e0-bf7b-4653-ad88-6ca949c4c02b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.922976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91f72e-c465-49b9-a0b6-2c7f4de1eb7d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.931018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 876/1765] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:58 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:58.931287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.932186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.932520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.932805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] PATH_INFO: `/v3/projects/3917698c574d4f52a6cb662c58b255e5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.933662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee39fb-ce04-4a19-a87d-394a11b5eee1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.934165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee39fb-ce04-4a19-a87d-394a11b5eee1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.934790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee39fb-ce04-4a19-a87d-394a11b5eee1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.935412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3917698c574d4f52a6cb662c58b255e5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.936906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 877/1766] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:02:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:02:58.937737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee39fb-ce04-4a19-a87d-394a11b5eee1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:58.939443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.963204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee39fb-ce04-4a19-a87d-394a11b5eee1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:58.970419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:58.971041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:58.971324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:58.971626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:58.972278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:58.973893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:58.987345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 878/1767] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:02:58.990992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.014883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317ad59-9151-4da8-ac9d-47c5d8d64128 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.029327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3917698c574d4f52a6cb662c58b255e5', 'request_id': 'req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.053545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3917698c574d4f52a6cb662c58b255e5', 'request_id': 'req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.070697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3917698c574d4f52a6cb662c58b255e5', 'request_id': 'req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.090644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] The token cache is being invalidate because project 3917698c574d4f52a6cb662c58b255e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:02:59.091000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.091790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d654bc0-900a-4dbc-924d-9b2c8a0bf593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.093640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 890/1768] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:02:58 2022] DELETE /identity/v3/projects/3917698c574d4f52a6cb662c58b255e5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:02:59.095166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.095550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.095863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.124965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:59.127097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:59.144341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:59.145085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.145382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.145942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.147749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:59.149963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:59.162025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 879/1769] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:02:59.163048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.169639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:59.189681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa37df8-1a1a-4b4e-a87d-07be36be85b6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.195615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:59.196878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.197152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.197422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] PATH_INFO: `/v3/users/ac66169770474696a9bfcd1963e1f209` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.198720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac66169770474696a9bfcd1963e1f209)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:59.200656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:59.234381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 891/1770] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:02:59.235543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.237134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:59.252842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9472d-0029-4634-8f8f-e065cdc27d18 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.258365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99818038-fad8-4636-9b01-8dd61bc9f7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:59.259578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99818038-fad8-4636-9b01-8dd61bc9f7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.259916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99818038-fad8-4636-9b01-8dd61bc9f7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.260192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99818038-fad8-4636-9b01-8dd61bc9f7a7 admin admin] PATH_INFO: `/v3/projects/2abf5c7de3194fb7b7a1a68123f0c36f/users/22dcd576eb5443fbb1f8381e6210b7e6/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.262862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:59.267475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac66169770474696a9bfcd1963e1f209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.282064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99818038-fad8-4636-9b01-8dd61bc9f7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abf5c7de3194fb7b7a1a68123f0c36f, user_id=22dcd576eb5443fbb1f8381e6210b7e6, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:59.283027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac66169770474696a9bfcd1963e1f209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.283984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac66169770474696a9bfcd1963e1f209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.286238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99818038-fad8-4636-9b01-8dd61bc9f7a7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:59.295077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac66169770474696a9bfcd1963e1f209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.296762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:02:59.305007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 892/1771] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:02:59 2022] PUT /identity/v3/projects/2abf5c7de3194fb7b7a1a68123f0c36f/users/22dcd576eb5443fbb1f8381e6210b7e6/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:59.316055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.316469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.316787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.330406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac66169770474696a9bfcd1963e1f209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:02:59.338099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 880/1772] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:02:59 2022] DELETE /identity/v3/users/ac66169770474696a9bfcd1963e1f209 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:02:59.353207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] MFA Rules not processed for user `22dcd576eb5443fbb1f8381e6210b7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:02:59.354871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:59.455787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 893/1773] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:02:59.457251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.458683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:59.597920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.602119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:02:59.639909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac2eb6-570f-4708-8f66-2ab092d833f4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.646316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:59.647161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.647618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.648040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.649687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:59.652284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:59.700344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 881/1774] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:02:59 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:59.706675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.785701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1fef1-2bd4-44ad-af56-e54faa1df81f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.871838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052f0ae-32fd-4ace-8d57-4dd84cac7402 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.880921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:59.881701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.882990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.884664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.886858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:02:59.890878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:02:59.948041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 882/1775] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:02:59 2022] POST /identity/v3/users => generated 296 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:02:59.952567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:02:59.991089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:02:59.992058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:02:59.992058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:02:59.992366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:02:59.994525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc1263-a797-4358-9e15-67cded736029 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:02:59.996877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:00.009719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:00.010776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.011071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.011338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.012112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.013683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.020499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:00.027785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.029548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.039265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 883/1776] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:00.040514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:00.050599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 894/1777] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:02:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:00.077003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff77733-3c6d-46bf-bb9d-01b4aaa79b4e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:00.091007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:00.091622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.091980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.092285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] PATH_INFO: `/v3/projects/bda356d921cb4090b5e7ed95c692d9b6/users/5d088f01814c4d18b1a80e40d5d682ee/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.116123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda356d921cb4090b5e7ed95c692d9b6, user_id=5d088f01814c4d18b1a80e40d5d682ee, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.120487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.147205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 884/1778] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:00 2022] PUT /identity/v3/projects/bda356d921cb4090b5e7ed95c692d9b6/users/5d088f01814c4d18b1a80e40d5d682ee/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:00.152077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:00.160357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:00.169774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:00.173815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ebbe9-b456-48b6-b027-dce1ef3af832 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:00.224263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:00.359026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:00.359777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.359987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.360430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.361125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.363039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.404528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c4ef-736c-4c02-b2e1-36ad927df325 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:00.416911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:00.417931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.418562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.419084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] PATH_INFO: `/v3/projects/bdfa2917fd6b4b639c9b86e4714529c5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.420807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfa2917fd6b4b639c9b86e4714529c5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.423933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.424513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 895/1779] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:00.428590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:00.472378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4c11d-1768-4848-937e-ead3cda0fd9d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:00.487551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9791a-5aa3-4921-9c6f-d7ba2af85f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:00.488625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9791a-5aa3-4921-9c6f-d7ba2af85f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.488768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9791a-5aa3-4921-9c6f-d7ba2af85f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.489003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9791a-5aa3-4921-9c6f-d7ba2af85f42 admin admin] PATH_INFO: `/v3/projects/bda356d921cb4090b5e7ed95c692d9b6/users/5d088f01814c4d18b1a80e40d5d682ee/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.494751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9791a-5aa3-4921-9c6f-d7ba2af85f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda356d921cb4090b5e7ed95c692d9b6, user_id=5d088f01814c4d18b1a80e40d5d682ee, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.502564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9791a-5aa3-4921-9c6f-d7ba2af85f42 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.541448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 896/1780] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:00 2022] PUT /identity/v3/projects/bda356d921cb4090b5e7ed95c692d9b6/users/5d088f01814c4d18b1a80e40d5d682ee/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:00.543930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.544188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.544467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.555695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfa2917fd6b4b639c9b86e4714529c5', 'request_id': 'req-dd660953-dae1-44b8-af73-2e61cabbc638'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:00.577968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfa2917fd6b4b639c9b86e4714529c5', 'request_id': 'req-dd660953-dae1-44b8-af73-2e61cabbc638'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:00.598898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfa2917fd6b4b639c9b86e4714529c5', 'request_id': 'req-dd660953-dae1-44b8-af73-2e61cabbc638'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:00.613146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] MFA Rules not processed for user `5d088f01814c4d18b1a80e40d5d682ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:00.613919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:00.645359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] The token cache is being invalidate because project bdfa2917fd6b4b639c9b86e4714529c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:00.648250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:00.650077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dd660953-dae1-44b8-af73-2e61cabbc638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:00.654567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 885/1781] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:00 2022] DELETE /identity/v3/projects/bdfa2917fd6b4b639c9b86e4714529c5 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:00.670448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb16fd-67c9-4bc3-83db-a950a43eabcb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.672688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb16fd-67c9-4bc3-83db-a950a43eabcb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.673239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb16fd-67c9-4bc3-83db-a950a43eabcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.724437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 897/1782] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:00.740401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:00.742464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:00.757146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-effb16fd-67c9-4bc3-83db-a950a43eabcb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:00.760907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-effb16fd-67c9-4bc3-83db-a950a43eabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:00.868949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 886/1783] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:00.889256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.889578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.891242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.920991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad986f4-cf4b-4dd5-b2e5-cde4598fd289 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:00.939513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:00.940474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:00.940927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:00.941427 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:00.959722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:00.964482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:00.966057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:00.968051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:00.999679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 887/1784] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:00.999679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:00.999966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 898/1785] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:01.074991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4b36-fffc-4c63-978b-f2a5643fe125 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:01.114961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:01.117341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.118058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.119046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:01.119829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:01.124845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:01.127118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:01.190970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 888/1786] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:01.196001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:01.220240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c291-c6b6-4d1e-80b0-749346657b43 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:01.225861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:01.226729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.227052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.227533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:01.228613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:01.231781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:01.238345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:01.318953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:01.327241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 899/1787] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:01 2022] POST /identity/v3/projects => generated 391 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:01.333675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a04e10-901c-4afe-925b-af6197b1e093 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:01.429367 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:01.476958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:01.478252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.478743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.479222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:01.480342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:01.482599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:01.543647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 889/1788] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:01 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:01.547980 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:01.566980 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcb55-0bc5-41d8-b802-ae5d24147cb0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:01.577070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:01.577712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.577989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.578258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:01.582366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:01.587889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:01.626392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 900/1789] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:01.628128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:01.660706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae83d6-ef86-4e09-97f5-340ea2e843b1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:01.680230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192780e5-ce41-4000-8f5b-c492060b6ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:01.681677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192780e5-ce41-4000-8f5b-c492060b6ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.681968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192780e5-ce41-4000-8f5b-c492060b6ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.682466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192780e5-ce41-4000-8f5b-c492060b6ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:01.683426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192780e5-ce41-4000-8f5b-c492060b6ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:01.685361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192780e5-ce41-4000-8f5b-c492060b6ff7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:01.701156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 890/1790] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:01.705711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:01.738046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c483f-0672-4c1d-9d1b-e25153653cf8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:01.772139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:01.772797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.773719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.774839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] PATH_INFO: `/v3/projects/cac61ebb2da6404086c6294f76ece4aa/users/2836f4600c69460fa21fa244d9b887a4/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:01.854222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac61ebb2da6404086c6294f76ece4aa, user_id=2836f4600c69460fa21fa244d9b887a4, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:01.857252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:01.879724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 901/1791] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:01 2022] PUT /identity/v3/projects/cac61ebb2da6404086c6294f76ece4aa/users/2836f4600c69460fa21fa244d9b887a4/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:01.893458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:01.893844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:01.894224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:02.009869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] MFA Rules not processed for user `2836f4600c69460fa21fa244d9b887a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:02.017451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:02.033505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.283442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a2326-ee25-4439-a6f6-e009f75dfc9c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:02.314072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d088f01814c4d18b1a80e40d5d682ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bda356d921cb4090b5e7ed95c692d9b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:02.317954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 891/1792] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:02.321914 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:02.323833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.326217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:02.326777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:02.328480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:02.333737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:02.343178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:02.428089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 902/1793] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:03:02.431985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.478962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b192d4-29a7-420c-a09a-dd0459d434ca tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:02.492592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d088f01814c4d18b1a80e40d5d682ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bda356d921cb4090b5e7ed95c692d9b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:02.497013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:02.498087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:02.499659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:02.500627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:02.509136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:02.544914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a850a-1f36-4d83-a8da-a1a6cab247c3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:02.557341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:02.560529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:02.560840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:02.561838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:02.564195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:02.569852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:02.597754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 903/1794] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:02 2022] POST /identity/v3/projects => generated 355 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:02.601671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.660272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 892/1795] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:02 2022] POST /identity/v3/projects => generated 391 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:02.664503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.748789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:02.845369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2534a2-0407-459a-bfdd-5e8ee8fe6714 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:02.857164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:02.861206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:02.861521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:02.861797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:02.862465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:02.865644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:02.878797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f628-cdae-4a21-a18d-46279af15bc8 tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:02.904228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 893/1796] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:02 2022] POST /identity/v3/users => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:02.909342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.934205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6373ce9-fc6d-4758-bf2a-2ce18de6ce4e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:02.939518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:02.940187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:02.940467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:02.942827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:02.945516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:02.946410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:02.947723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:02.981247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 894/1797] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:02.982203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:02.998201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc42e3-b8fb-4f1d-a59c-fe8b8810a713 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:03.015188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d088f01814c4d18b1a80e40d5d682ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bda356d921cb4090b5e7ed95c692d9b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:03.015851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.016179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.016901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:03.017639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.017842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.018170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] PATH_INFO: `/v3/projects/1493bc09a4464b3285821f2ae1be6122/users/e1c5f18b4deb4227a5ae0f4e13cc62f6/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.020418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.020418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:03.020627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:03.033400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:03.070306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1493bc09a4464b3285821f2ae1be6122, user_id=e1c5f18b4deb4227a5ae0f4e13cc62f6, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:03.072660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 904/1798] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:03.091128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:03.126182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 895/1799] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:02 2022] PUT /identity/v3/projects/1493bc09a4464b3285821f2ae1be6122/users/e1c5f18b4deb4227a5ae0f4e13cc62f6/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:03.130385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:03.146399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4987713-0fbe-49e7-80ec-a16edbd86fbb tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:03.160621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:03.360835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:03.361434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.361706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.361986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.362799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:03.375515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:03.378562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee507f-7c65-4e8f-a43b-7087d75aaa77 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:03.395678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d088f01814c4d18b1a80e40d5d682ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bda356d921cb4090b5e7ed95c692d9b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:03.397908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.398197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.398470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] PATH_INFO: `/v3/projects/bf9a23271f644298984826bca439a3a0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.405171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bf9a23271f644298984826bca439a3a0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:03.409417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:03.421957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 905/1800] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:03.422912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:03.458594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdca7c-91ed-4f1b-9500-a7a843de7060 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:03.475245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f348a-d50c-4e24-a3a0-3a5256fe2af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:03.476949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f348a-d50c-4e24-a3a0-3a5256fe2af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.477555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f348a-d50c-4e24-a3a0-3a5256fe2af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.477555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f348a-d50c-4e24-a3a0-3a5256fe2af1 admin admin] PATH_INFO: `/v3/projects/1493bc09a4464b3285821f2ae1be6122/users/e1c5f18b4deb4227a5ae0f4e13cc62f6/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.488157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f348a-d50c-4e24-a3a0-3a5256fe2af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1493bc09a4464b3285821f2ae1be6122, user_id=e1c5f18b4deb4227a5ae0f4e13cc62f6, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:03.490989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f348a-d50c-4e24-a3a0-3a5256fe2af1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:03.536110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 906/1801] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:03 2022] PUT /identity/v3/projects/1493bc09a4464b3285821f2ae1be6122/users/e1c5f18b4deb4227a5ae0f4e13cc62f6/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:03.559161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.559486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.559812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.584971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9a23271f644298984826bca439a3a0', 'request_id': 'req-b50601bb-88cd-4a6d-9475-5a05fda8206e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:03.599316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9a23271f644298984826bca439a3a0', 'request_id': 'req-b50601bb-88cd-4a6d-9475-5a05fda8206e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:03.615964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9a23271f644298984826bca439a3a0', 'request_id': 'req-b50601bb-88cd-4a6d-9475-5a05fda8206e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:03.623094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] MFA Rules not processed for user `e1c5f18b4deb4227a5ae0f4e13cc62f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:03.624364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:03.637328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] The token cache is being invalidate because project bf9a23271f644298984826bca439a3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:03.637794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:03.638826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:03.646753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 896/1802] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:03 2022] DELETE /identity/v3/projects/bf9a23271f644298984826bca439a3a0 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:03.748370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 907/1803] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:03.792794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:03.794316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:03.827180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:03.962600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50601bb-88cd-4a6d-9475-5a05fda8206e tempest-VolumeQuotaClassesTest-729156097 tempest-VolumeQuotaClassesTest-729156097-project-admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:03.971046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:03.972157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:03.972607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:03.973025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:03.990428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:03.992555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:04.010121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 897/1804] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:04.141047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:04.157003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef7b7f-9609-43cd-88b5-6e673a88199b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:04.174887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:04.175315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:04.175590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:04.175927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:04.185380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:04.205792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:04.207238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:04.283629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:04.365050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:04.367847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85171-62c5-41f3-a7f2-2d65f8fbb930 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:04.376183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb0e6e-a008-439e-a7d4-784d5972a11c service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:04.391390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:04.392097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:04.392377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:04.392647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] PATH_INFO: `/v3/users/22dcd576eb5443fbb1f8381e6210b7e6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:04.396008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 908/1805] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:04.408315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=22dcd576eb5443fbb1f8381e6210b7e6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:04.410591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:04.539038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22dcd576eb5443fbb1f8381e6210b7e6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:04.548779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22dcd576eb5443fbb1f8381e6210b7e6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:04.549522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22dcd576eb5443fbb1f8381e6210b7e6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:04.563204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22dcd576eb5443fbb1f8381e6210b7e6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:04.638387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22dcd576eb5443fbb1f8381e6210b7e6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:04.661603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:04.662300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:04.662697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:04.672999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 898/1806] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:04 2022] DELETE /identity/v3/users/22dcd576eb5443fbb1f8381e6210b7e6 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:04.689614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:04.691158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:04.772099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 909/1807] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:04.913692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:04.916225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:05.000112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:05.033474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a814-caea-473f-8106-a330c2c4bd3f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:05.037795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:05.038600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:05.038978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:05.039419 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:05.050567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:05.052395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:05.067364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 899/1808] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:05.585547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:05.589296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:05.610140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f5a31-2daa-45b2-ae63-7a6a3d76d324 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:05.622947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:05.623806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:05.624220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:05.624571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] PATH_INFO: `/v3/projects/2abf5c7de3194fb7b7a1a68123f0c36f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:05.626947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abf5c7de3194fb7b7a1a68123f0c36f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:05.629219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:05.685079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abf5c7de3194fb7b7a1a68123f0c36f', 'request_id': 'req-db88573b-a29e-4f3e-a4d8-47abc3afd72d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.693457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abf5c7de3194fb7b7a1a68123f0c36f', 'request_id': 'req-db88573b-a29e-4f3e-a4d8-47abc3afd72d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.702105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abf5c7de3194fb7b7a1a68123f0c36f', 'request_id': 'req-db88573b-a29e-4f3e-a4d8-47abc3afd72d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.714420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] The token cache is being invalidate because project 2abf5c7de3194fb7b7a1a68123f0c36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:05.714763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.715450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.717056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 910/1809] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:05 2022] DELETE /identity/v3/projects/2abf5c7de3194fb7b7a1a68123f0c36f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:05.721228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:05.733340 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519abbbd-bd3f-4f49-b699-a5516fb809f7 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:05.822839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:05.823489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:05.823802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:05.824072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] PATH_INFO: `/v3/users/5d088f01814c4d18b1a80e40d5d682ee` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:05.825228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d088f01814c4d18b1a80e40d5d682ee)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:05.827165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:05.898440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d088f01814c4d18b1a80e40d5d682ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.908294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d088f01814c4d18b1a80e40d5d682ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.909183 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d088f01814c4d18b1a80e40d5d682ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.919365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d088f01814c4d18b1a80e40d5d682ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.947144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3bf9817b-e34a-44cd-bfae-5eb4a37fa572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d088f01814c4d18b1a80e40d5d682ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:05.959641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 900/1810] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:05 2022] DELETE /identity/v3/users/5d088f01814c4d18b1a80e40d5d682ee => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:07.364424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:07.369970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:07.391960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88573b-a29e-4f3e-a4d8-47abc3afd72d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:07.489231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:07.507144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:07.508065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:07.509340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:07.509618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] PATH_INFO: `/v3/projects/bda356d921cb4090b5e7ed95c692d9b6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:07.510981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda356d921cb4090b5e7ed95c692d9b6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:07.513093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:07.574491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda356d921cb4090b5e7ed95c692d9b6', 'request_id': 'req-4932bc45-1365-4e99-9c87-a6cf6ff59263'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:07.612295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda356d921cb4090b5e7ed95c692d9b6', 'request_id': 'req-4932bc45-1365-4e99-9c87-a6cf6ff59263'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:07.634846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda356d921cb4090b5e7ed95c692d9b6', 'request_id': 'req-4932bc45-1365-4e99-9c87-a6cf6ff59263'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:07.656300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] The token cache is being invalidate because project bda356d921cb4090b5e7ed95c692d9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:07.656861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:07.657633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4932bc45-1365-4e99-9c87-a6cf6ff59263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:07.659707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 911/1811] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:07 2022] DELETE /identity/v3/projects/bda356d921cb4090b5e7ed95c692d9b6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:07.664514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:07.665549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:07.665990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:07.692192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:07.693861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:07.776640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:07.791955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 901/1812] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:07.800668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:07.801128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:07.801435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:07.828492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:07.829500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:07.850996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 912/1813] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:07.858410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:07.874855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34995a-fa1f-421f-bad5-ecde1bd116bf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:07.880949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:07.881583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:07.881906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:07.882220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:07.882870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:07.883171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:07.887181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:07.924369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 902/1814] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:07.925921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:07.935554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f9df3-a2f8-4b55-b738-66121bad6eb5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:07.957946 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:07.958583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:07.958864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:07.959138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:07.959978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:07.962458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.018839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 913/1815] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:07 2022] POST /identity/v3/projects => generated 407 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:08.023182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.035755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacecc21-6cff-429f-a016-7364d793ce90 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:08.190270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:08.201899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.202590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.202590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.203140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:08.212212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.268155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 903/1816] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:08 2022] POST /identity/v3/users => generated 304 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:08.272374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.291245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28067fcd-9a00-4589-9077-777d57f2365b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:08.299678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:08.300316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.300603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.300925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.301610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:08.303245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.325892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 914/1817] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:08.328754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.355116 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c84009-2637-41aa-8bbd-d9228c029e83 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:08.370497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:08.371131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.371407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.371815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.372362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:08.374023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.425809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.429093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 904/1818] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:08.450832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0323ad-59e5-42b7-86f6-4fb4e40d3ec7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:08.468218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eafe6-91df-4d0c-b597-1161d4c13ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:08.470676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eafe6-91df-4d0c-b597-1161d4c13ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.471432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eafe6-91df-4d0c-b597-1161d4c13ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.482303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eafe6-91df-4d0c-b597-1161d4c13ff0 admin admin] PATH_INFO: `/v3/projects/a223481a0c3343e8a5c97f7bd7a2d68b/users/a04131c09c454436a61a1c0763a43dcc/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.516028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eafe6-91df-4d0c-b597-1161d4c13ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a223481a0c3343e8a5c97f7bd7a2d68b, user_id=a04131c09c454436a61a1c0763a43dcc, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:08.523066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eafe6-91df-4d0c-b597-1161d4c13ff0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.547117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.549105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:08.556528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 915/1819] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:08 2022] PUT /identity/v3/projects/a223481a0c3343e8a5c97f7bd7a2d68b/users/a04131c09c454436a61a1c0763a43dcc/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:08.572308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.572947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.573228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.605617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5dc2a-d454-4181-9e68-1b0617c3e5dd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:08.653752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] MFA Rules not processed for user `a04131c09c454436a61a1c0763a43dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:08.657241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:08.780453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:08.781114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.781398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.781674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] PATH_INFO: `/v3/users/2836f4600c69460fa21fa244d9b887a4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.785261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2836f4600c69460fa21fa244d9b887a4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:08.792193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.840649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 916/1820] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:08 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:08.841790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.857587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c9a3-9f70-41a8-b20d-39c4e6ada46b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:08.864366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:08.864988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.865270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.865588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:08.866605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:08.873984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:08.885521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:08.902629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2836f4600c69460fa21fa244d9b887a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:08.918511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2836f4600c69460fa21fa244d9b887a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:08.919373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2836f4600c69460fa21fa244d9b887a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:08.924969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:08.933582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2836f4600c69460fa21fa244d9b887a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:08.936465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 917/1821] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:08 2022] POST /identity/v3/projects => generated 407 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:08.942000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:08.946753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:08.956269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0e581d82-a215-49ee-bef4-7df10a34471c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2836f4600c69460fa21fa244d9b887a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:08.972111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 905/1822] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:08 2022] DELETE /identity/v3/users/2836f4600c69460fa21fa244d9b887a4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:08.990518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:08.990822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:08.991114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.037049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:09.038940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:09.056764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c7ab6-6300-40cc-9a24-bf7084bb24c2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.163138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.163823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.164178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.164774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.165541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.168750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.205730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 906/1823] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:09.214127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 918/1824] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:08 2022] POST /identity/v3/users => generated 310 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:09.217300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:09.217866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:09.219092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:09.238535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a223f-222f-4c0e-bb1d-4e48cd8b4087 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.254041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.255057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.255368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.255666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.256331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.259943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.283167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 919/1825] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:09.287184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:09.293236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572f96-f47d-4cbf-bdec-19e60487e1df None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.306138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ec82b-221d-4f61-895c-84f26b2cc95f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.319048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.321181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.321682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.322809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.323560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.323690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.323957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.324323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] PATH_INFO: `/v3/projects/00386e4803454167b4c814b54100c20a/users/6f003a12d8ca405ba5465ce9bdbf9d91/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.346796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.357151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00386e4803454167b4c814b54100c20a, user_id=6f003a12d8ca405ba5465ce9bdbf9d91, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.362955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.366052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.384931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 907/1826] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:09.397978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 920/1827] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:09 2022] PUT /identity/v3/projects/00386e4803454167b4c814b54100c20a/users/6f003a12d8ca405ba5465ce9bdbf9d91/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:09.403330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:09.414306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5265e-6d04-4533-8c8f-fb656049131a service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.500198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3238d-11f2-4fdf-a23d-c2a9874b959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.500836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3238d-11f2-4fdf-a23d-c2a9874b959a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.501112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3238d-11f2-4fdf-a23d-c2a9874b959a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.501615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3238d-11f2-4fdf-a23d-c2a9874b959a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.502791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3238d-11f2-4fdf-a23d-c2a9874b959a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.505940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3238d-11f2-4fdf-a23d-c2a9874b959a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.522603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 908/1828] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:09.526962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:09.542225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b967740-7e27-4e2c-86e8-68f52ca1ecf6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.548154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.548836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.549620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.550209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] PATH_INFO: `/v3/projects/00386e4803454167b4c814b54100c20a/users/6f003a12d8ca405ba5465ce9bdbf9d91/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.552620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00386e4803454167b4c814b54100c20a, user_id=6f003a12d8ca405ba5465ce9bdbf9d91, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.556464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.587383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.587779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.588137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.592791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 921/1829] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:09 2022] PUT /identity/v3/projects/00386e4803454167b4c814b54100c20a/users/6f003a12d8ca405ba5465ce9bdbf9d91/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:09.681692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] MFA Rules not processed for user `6f003a12d8ca405ba5465ce9bdbf9d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:09.682933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:09.697095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:09.804554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 909/1830] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:09.816145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:09.818185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:09.866600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:09.887113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5cae9-4891-4e59-b176-6624fbc656a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:09.890746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:09.891354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:09.891658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:09.891932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:09.901879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:09.903347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:09.913049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 922/1831] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:10.087292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:10.089624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:10.155666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:10.169071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8515a-f922-4f29-8f1d-695fc7648493 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:10.174155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:10.174913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:10.175314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:10.175702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:10.177820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:10.226755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2132be-df42-4d3b-9cb3-d815663043d3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:10.235458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:10.236499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:10.237202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:10.238439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] PATH_INFO: `/v3/projects/cac61ebb2da6404086c6294f76ece4aa` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:10.242876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac61ebb2da6404086c6294f76ece4aa)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:10.249298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:10.299459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:10.300965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:10.321271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 923/1832] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:10 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:10.356703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac61ebb2da6404086c6294f76ece4aa', 'request_id': 'req-9a71e2f9-29af-47be-a5d7-d4ab771ea790'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.377236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac61ebb2da6404086c6294f76ece4aa', 'request_id': 'req-9a71e2f9-29af-47be-a5d7-d4ab771ea790'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.399709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac61ebb2da6404086c6294f76ece4aa', 'request_id': 'req-9a71e2f9-29af-47be-a5d7-d4ab771ea790'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.436200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] The token cache is being invalidate because project cac61ebb2da6404086c6294f76ece4aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:10.436556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.438395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9a71e2f9-29af-47be-a5d7-d4ab771ea790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.443421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 910/1833] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:10 2022] DELETE /identity/v3/projects/cac61ebb2da6404086c6294f76ece4aa => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:10.448230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:10.473584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e796b36-7b80-4bf3-8210-942ba5455df1 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:10.569895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:10.570696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:10.570947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:10.571392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] PATH_INFO: `/v3/users/e1c5f18b4deb4227a5ae0f4e13cc62f6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:10.572904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c5f18b4deb4227a5ae0f4e13cc62f6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:10.576334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:10.664533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c5f18b4deb4227a5ae0f4e13cc62f6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.685120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c5f18b4deb4227a5ae0f4e13cc62f6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.703507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c5f18b4deb4227a5ae0f4e13cc62f6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.706960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c5f18b4deb4227a5ae0f4e13cc62f6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.733739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:10.735307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-346e81c0-6936-4afa-a43e-86eff5aa64c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c5f18b4deb4227a5ae0f4e13cc62f6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:10.747141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 924/1834] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:10 2022] DELETE /identity/v3/users/e1c5f18b4deb4227a5ae0f4e13cc62f6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:10.834879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132da2-73ec-4c09-b1d6-5988a03eefb8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:10.834879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132da2-73ec-4c09-b1d6-5988a03eefb8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:10.834879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132da2-73ec-4c09-b1d6-5988a03eefb8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:10.844619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 911/1835] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:03:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:03:10.902684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:10.903030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:10.903307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:10.959055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:10.960391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:11.158327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 925/1836] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:11.190996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132da2-73ec-4c09-b1d6-5988a03eefb8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:11.193430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ee132da2-73ec-4c09-b1d6-5988a03eefb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:11.250158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132da2-73ec-4c09-b1d6-5988a03eefb8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:11.270383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2c024-f28c-4d33-b646-ac768ffcc27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:11.270997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2c024-f28c-4d33-b646-ac768ffcc27f service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:11.271667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2c024-f28c-4d33-b646-ac768ffcc27f service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:11.271989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2c024-f28c-4d33-b646-ac768ffcc27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:11.301283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2c024-f28c-4d33-b646-ac768ffcc27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:11.308079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2c024-f28c-4d33-b646-ac768ffcc27f service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:11.335926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 912/1837] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:11.559056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:11.560985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:11.641373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8018c1-290b-4a95-91c8-e632da838bd6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:11.649349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:11.650589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:11.651060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:11.651340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] PATH_INFO: `/v3/projects/1493bc09a4464b3285821f2ae1be6122` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:11.652658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1493bc09a4464b3285821f2ae1be6122)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:11.655670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:11.708557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1493bc09a4464b3285821f2ae1be6122', 'request_id': 'req-29a70073-691b-416c-a8b5-be804ffc119b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:11.716444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1493bc09a4464b3285821f2ae1be6122', 'request_id': 'req-29a70073-691b-416c-a8b5-be804ffc119b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:11.722910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1493bc09a4464b3285821f2ae1be6122', 'request_id': 'req-29a70073-691b-416c-a8b5-be804ffc119b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:11.734429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] The token cache is being invalidate because project 1493bc09a4464b3285821f2ae1be6122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:11.734717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:11.735409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-29a70073-691b-416c-a8b5-be804ffc119b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:11.737375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 926/1838] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:11 2022] DELETE /identity/v3/projects/1493bc09a4464b3285821f2ae1be6122 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:11.742378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:11.743171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:11.743451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:11.766097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:11.768001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:11.788362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:11.860110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 913/1839] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:11.863750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:11.863750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:11.863750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:11.885582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:11.886834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:11.900064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 927/1840] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:11.900843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:11.911694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369d3b-b51a-4978-8c54-bb155348d6ac None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:11.917289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:11.918358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:11.918358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:11.918554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:11.919279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:11.919677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:11.921349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:11.947490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 914/1841] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:03:11.951940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:11.963023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaedc5-0c4f-44b7-926c-9696f1c69dbf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:11.968272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:11.968891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:11.969355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:11.969640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:11.970781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:11.972853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.005389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:12.049515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 928/1842] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:11 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:12.051027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.068863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d739-8bae-47f9-ab8b-d9b2f40064f7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.153404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.154447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.154748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.155153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.155960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.157983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.228470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 915/1843] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:12 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:12.230742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.244467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ea61-2bd8-4d45-9380-552f959fb58b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.250371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.251491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.252113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.252618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.253538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.255994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.271827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 929/1844] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:12.276126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.287159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad79c3-3355-4fcd-937b-fb5891a3fc66 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.293817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71971b-dd43-4e7b-babe-2f131690e84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.304903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71971b-dd43-4e7b-babe-2f131690e84b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.305199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71971b-dd43-4e7b-babe-2f131690e84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.305662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71971b-dd43-4e7b-babe-2f131690e84b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.306409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71971b-dd43-4e7b-babe-2f131690e84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.308818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71971b-dd43-4e7b-babe-2f131690e84b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.329750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 916/1845] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:12.333955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.345523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c672fa-075b-44ca-bf60-27f7b449c3a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.351275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.352584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.352866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.353147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] PATH_INFO: `/v3/projects/d270a84081b741fcbffd3125d6906472/users/a45cdb2136a44edab8e606196f7d9d3d/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.370874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d270a84081b741fcbffd3125d6906472, user_id=a45cdb2136a44edab8e606196f7d9d3d, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.373757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.400849 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 930/1846] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:12 2022] PUT /identity/v3/projects/d270a84081b741fcbffd3125d6906472/users/a45cdb2136a44edab8e606196f7d9d3d/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:12.412401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.413016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.413509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.451423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] MFA Rules not processed for user `a45cdb2136a44edab8e606196f7d9d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:12.453443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:12.549844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 917/1847] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:12.554182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.636390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17e058-df8c-4f2e-ad4e-768268e4df15 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.643825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.644459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.644714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.644996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.646013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.649771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.686239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 931/1848] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:12 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:12.690019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.707787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cefdc-c8f0-43d4-ab37-de4cb45aa0ad None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.719891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:12.767317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.767964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.768308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.768586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.769265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.771172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.794829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 918/1849] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:12 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:12.798811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.811591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9850ff7-1281-4c32-b845-a265351b41ae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.816858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.817573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.817883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.818159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.819053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.822793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.840366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 932/1850] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:12.840687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.852312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705dd71-cf1b-451b-94ce-6bdf2f40c483 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.857040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.857738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.857990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.858264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] PATH_INFO: `/v3/projects/a0bc2e39e0624c2cac41d864e7e84b7d/users/5c2d26972fd744dba27e91e7534dc4b7/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.876718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0bc2e39e0624c2cac41d864e7e84b7d, user_id=5c2d26972fd744dba27e91e7534dc4b7, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.879690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.894586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 919/1851] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:12 2022] PUT /identity/v3/projects/a0bc2e39e0624c2cac41d864e7e84b7d/users/5c2d26972fd744dba27e91e7534dc4b7/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:12.898712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.909363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd28f04-e547-4787-836d-4560aa2af980 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:12.968866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df4f0e-bd75-46d6-b4b6-8c75eca823c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:12.969485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df4f0e-bd75-46d6-b4b6-8c75eca823c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:12.969766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df4f0e-bd75-46d6-b4b6-8c75eca823c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:12.970039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df4f0e-bd75-46d6-b4b6-8c75eca823c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:12.970718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df4f0e-bd75-46d6-b4b6-8c75eca823c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:12.972344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df4f0e-bd75-46d6-b4b6-8c75eca823c6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:12.982919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 933/1852] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:12.986913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:12.998195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4fdb-3267-4506-8014-215ab432cc99 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:13.002846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:13.003576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:13.003931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:13.004541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] PATH_INFO: `/v3/projects/a0bc2e39e0624c2cac41d864e7e84b7d/users/5c2d26972fd744dba27e91e7534dc4b7/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:13.006578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0bc2e39e0624c2cac41d864e7e84b7d, user_id=5c2d26972fd744dba27e91e7534dc4b7, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:13.010148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:13.021368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 920/1853] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:12 2022] PUT /identity/v3/projects/a0bc2e39e0624c2cac41d864e7e84b7d/users/5c2d26972fd744dba27e91e7534dc4b7/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:13.031731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:13.032239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:13.032347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:13.067727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] MFA Rules not processed for user `5c2d26972fd744dba27e91e7534dc4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:13.068948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:13.081931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:13.140793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 934/1854] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:13.152844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:13.154527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:13.212050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed053f22-eb1b-46ec-95c4-2f9a346d4161 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:13.215715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:13.216325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:13.216609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:13.217702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:13.234123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:13.235577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:13.245976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 921/1855] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:13.334368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:13.345109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979987-2a8f-4284-9f71-326069021b94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:13.350124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d595ac-7fa3-4c2c-97e6-7119e7c059bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:13.350720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d595ac-7fa3-4c2c-97e6-7119e7c059bd service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:13.350995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d595ac-7fa3-4c2c-97e6-7119e7c059bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:13.351269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d595ac-7fa3-4c2c-97e6-7119e7c059bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:13.418890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d595ac-7fa3-4c2c-97e6-7119e7c059bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:13.420361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d595ac-7fa3-4c2c-97e6-7119e7c059bd service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:13.430921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 935/1856] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:13.710159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:13.771728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e291b69-1656-4179-8691-383590c9c2f0 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:13.776993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:13.777614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:13.777888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:13.778157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] PATH_INFO: `/v3/users/a45cdb2136a44edab8e606196f7d9d3d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:13.779267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45cdb2136a44edab8e606196f7d9d3d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:13.781201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:13.793979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:13.829066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45cdb2136a44edab8e606196f7d9d3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:13.836875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45cdb2136a44edab8e606196f7d9d3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:13.837663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45cdb2136a44edab8e606196f7d9d3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:13.851430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45cdb2136a44edab8e606196f7d9d3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:13.869976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45cdb2136a44edab8e606196f7d9d3d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:13.877285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 922/1857] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:13 2022] DELETE /identity/v3/users/a45cdb2136a44edab8e606196f7d9d3d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:13.882026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:13.882750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:13.882750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:13.904556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:13.905886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:13.971287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 936/1858] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:13.981883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:13.983500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:14.042018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424946-47a1-462f-8500-ec92edffd154 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:14.045662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:14.046211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:14.046487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:14.046755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:14.055905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:14.057327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:14.067098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 923/1859] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:14.412080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:14.413832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:14.428447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b276-6c63-4df7-af8d-3b7893036bf6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:14.434155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:14.434985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:14.435395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:14.435818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] PATH_INFO: `/v3/projects/d270a84081b741fcbffd3125d6906472` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:14.437055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d270a84081b741fcbffd3125d6906472)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:14.439220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:14.485917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:14.490091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd270a84081b741fcbffd3125d6906472', 'request_id': 'req-6e00230e-c06a-4a9c-b882-80e8dd209706'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.498513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd270a84081b741fcbffd3125d6906472', 'request_id': 'req-6e00230e-c06a-4a9c-b882-80e8dd209706'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.505544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd270a84081b741fcbffd3125d6906472', 'request_id': 'req-6e00230e-c06a-4a9c-b882-80e8dd209706'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.515948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] The token cache is being invalidate because project d270a84081b741fcbffd3125d6906472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:14.516242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.516896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.518619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 937/1860] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:14 2022] DELETE /identity/v3/projects/d270a84081b741fcbffd3125d6906472 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:14.522412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:14.533918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbcfde-b235-4d89-8c51-da34a738249a service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:14.599363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:14.600061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:14.600341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:14.600609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] PATH_INFO: `/v3/users/5c2d26972fd744dba27e91e7534dc4b7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:14.601767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2d26972fd744dba27e91e7534dc4b7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:14.604288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:14.643394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:14.657126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2d26972fd744dba27e91e7534dc4b7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.664148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2d26972fd744dba27e91e7534dc4b7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.665499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2d26972fd744dba27e91e7534dc4b7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.673143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2d26972fd744dba27e91e7534dc4b7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.686140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3c9e8298-946f-4b62-b1bd-91694fe65650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2d26972fd744dba27e91e7534dc4b7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:14.692268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 924/1861] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:14 2022] DELETE /identity/v3/users/5c2d26972fd744dba27e91e7534dc4b7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:15.064281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.065966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:15.081437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00230e-c06a-4a9c-b882-80e8dd209706 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.163573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.164356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.164866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.165270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] PATH_INFO: `/v3/projects/a0bc2e39e0624c2cac41d864e7e84b7d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.166416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0bc2e39e0624c2cac41d864e7e84b7d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.168353 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.218635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0bc2e39e0624c2cac41d864e7e84b7d', 'request_id': 'req-80846f68-b8f3-4594-8edb-cca572e0827b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:15.226073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0bc2e39e0624c2cac41d864e7e84b7d', 'request_id': 'req-80846f68-b8f3-4594-8edb-cca572e0827b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:15.232787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0bc2e39e0624c2cac41d864e7e84b7d', 'request_id': 'req-80846f68-b8f3-4594-8edb-cca572e0827b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:15.244826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] The token cache is being invalidate because project a0bc2e39e0624c2cac41d864e7e84b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:15.245116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:15.245776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80846f68-b8f3-4594-8edb-cca572e0827b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:15.247280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 938/1862] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:15 2022] DELETE /identity/v3/projects/a0bc2e39e0624c2cac41d864e7e84b7d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:15.252990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822cd99-6995-494b-b246-06149670eee6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.253487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822cd99-6995-494b-b246-06149670eee6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.253565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822cd99-6995-494b-b246-06149670eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.274896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3822cd99-6995-494b-b246-06149670eee6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:15.276160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3822cd99-6995-494b-b246-06149670eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:15.372157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 925/1863] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:15.385243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.385550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.385802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.429302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:15.432437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:15.454140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 939/1864] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:15.458073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822cd99-6995-494b-b246-06149670eee6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.471041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822cd99-6995-494b-b246-06149670eee6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.478996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.479811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.480193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.480589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.481428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:15.481834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.483660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.504153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 926/1865] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:15.505570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.516773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96116f5e-a9e7-4d01-b9d6-de746726a174 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.521589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.522204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.522562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.522886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.523460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.526412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.579369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 940/1866] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:15 2022] POST /identity/v3/projects => generated 359 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:15.581163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.598468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb7e2-f18e-4404-94a0-fee93518ed03 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.704326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.705093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.705714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.706002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.706817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.709639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.744349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 927/1867] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:15 2022] POST /identity/v3/users => generated 280 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:15.748331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.770374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03040-fc69-4905-a1ed-369ea122dc70 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.793463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.794245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.794504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.794782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.795437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.797066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.822800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 941/1868] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:15.824672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.846715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b256d7-427c-4903-8a32-08cae677c72a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.870166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb1042-1d1d-40b5-b7e2-ff275a31c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.870813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb1042-1d1d-40b5-b7e2-ff275a31c8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.871102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb1042-1d1d-40b5-b7e2-ff275a31c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.871379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb1042-1d1d-40b5-b7e2-ff275a31c8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.872578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb1042-1d1d-40b5-b7e2-ff275a31c8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.874217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb1042-1d1d-40b5-b7e2-ff275a31c8ad admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.899361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 928/1869] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:15.903578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:15.923081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363d3b-e0b3-4632-a6b2-e31d1e40297c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:15.932303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:15.933229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.933529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.933791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] PATH_INFO: `/v3/projects/0fdcf3e8030e4db79f877c867d558d1f/users/39575c00290d4bc5a74d2af726d6f50a/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:15.953691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fdcf3e8030e4db79f877c867d558d1f, user_id=39575c00290d4bc5a74d2af726d6f50a, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:15.956773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:15.968303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:15.982246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 942/1870] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:15 2022] PUT /identity/v3/projects/0fdcf3e8030e4db79f877c867d558d1f/users/39575c00290d4bc5a74d2af726d6f50a/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:15.996552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:15.997120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:15.997979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:16.060621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] MFA Rules not processed for user `39575c00290d4bc5a74d2af726d6f50a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:16.061983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:16.068054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:16.188049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 929/1871] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:16.227513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:16.229423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:16.300897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec628-1910-4cc6-97d9-996767659bc1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:16.304619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a822a2-108b-4b67-953d-8dbf4b149560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:16.311758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a822a2-108b-4b67-953d-8dbf4b149560 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:16.311758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a822a2-108b-4b67-953d-8dbf4b149560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:16.311758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a822a2-108b-4b67-953d-8dbf4b149560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:16.315886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a822a2-108b-4b67-953d-8dbf4b149560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:16.317628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a822a2-108b-4b67-953d-8dbf4b149560 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:16.326926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 943/1872] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:16.341610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:16.404567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00f935-73b5-4601-85a3-8b0f018322da None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:16.409049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:16.409675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:16.409948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:16.410390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] PATH_INFO: `/v3/users/39575c00290d4bc5a74d2af726d6f50a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:16.411513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=39575c00290d4bc5a74d2af726d6f50a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:16.413439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:16.458575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39575c00290d4bc5a74d2af726d6f50a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:16.466771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39575c00290d4bc5a74d2af726d6f50a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:16.467477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39575c00290d4bc5a74d2af726d6f50a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:16.475532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39575c00290d4bc5a74d2af726d6f50a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:16.487807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39575c00290d4bc5a74d2af726d6f50a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:16.494002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 930/1873] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:16 2022] DELETE /identity/v3/users/39575c00290d4bc5a74d2af726d6f50a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:16.498542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:16.498834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:16.499116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:16.519292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:16.520547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:16.579102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 944/1874] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:16.590121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:16.591895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:16.624095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:16.657584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93f8d7-a187-4bd8-8142-845e699fd24e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:16.661211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9482736-f712-400e-97bb-9a340801e6c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:16.662154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482736-f712-400e-97bb-9a340801e6c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:16.662423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482736-f712-400e-97bb-9a340801e6c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:16.662997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482736-f712-400e-97bb-9a340801e6c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:16.675996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9482736-f712-400e-97bb-9a340801e6c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:16.677440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9482736-f712-400e-97bb-9a340801e6c6 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:16.690196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 931/1875] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:17.375103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:17.376737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:17.425036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63983-c067-4a3d-8b55-62048c30004e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:17.433164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:17.435350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:17.436114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:17.436893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] PATH_INFO: `/v3/projects/0fdcf3e8030e4db79f877c867d558d1f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:17.439242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fdcf3e8030e4db79f877c867d558d1f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:17.442512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:17.600102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fdcf3e8030e4db79f877c867d558d1f', 'request_id': 'req-23503362-ea56-47d8-8322-524a7c5c6d38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:17.615585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fdcf3e8030e4db79f877c867d558d1f', 'request_id': 'req-23503362-ea56-47d8-8322-524a7c5c6d38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:17.625208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fdcf3e8030e4db79f877c867d558d1f', 'request_id': 'req-23503362-ea56-47d8-8322-524a7c5c6d38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:17.645323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] The token cache is being invalidate because project 0fdcf3e8030e4db79f877c867d558d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:17.645654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:17.646402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-23503362-ea56-47d8-8322-524a7c5c6d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:17.652256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 945/1876] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:17 2022] DELETE /identity/v3/projects/0fdcf3e8030e4db79f877c867d558d1f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:17.658013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:17.658628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:17.658950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:17.693097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:17.694665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:17.796417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 932/1877] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:17.807688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:17.808061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:17.808388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:17.848945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:17.850724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:17.876062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 946/1878] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:17.879775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:17.894467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e67c24-d28d-4684-b8e8-f90f3d4cf356 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:17.915144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:17.916548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:17.916908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:17.917181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:17.918277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:17.918659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:17.920420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:17.953911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 933/1879] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:17.962114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:17.973288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302db9-92ee-44a9-b85c-24cfbbdd8810 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:17.978630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:17.979314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:17.979680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:17.979957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:17.980594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:17.982569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.027800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:18.068023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 947/1880] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:17 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:18.072884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.084987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541cee2-d218-4eef-a003-e1aba6e1baa3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.186136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.187373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.187913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.188386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.189599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.192550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.251061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 934/1881] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:18 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:18.253005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.265317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2178a-627f-4709-944c-8607fe9a1d66 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.271831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.272616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.272905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.273183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.274577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.276357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.301029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 948/1882] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:18.308432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.321472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a910f3-3b58-44c1-952d-c90f8afadb5c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.326300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f321ee4-cfe5-425d-9571-742c1956175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.326954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f321ee4-cfe5-425d-9571-742c1956175c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.327343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f321ee4-cfe5-425d-9571-742c1956175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.327659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f321ee4-cfe5-425d-9571-742c1956175c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.328293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f321ee4-cfe5-425d-9571-742c1956175c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.329927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f321ee4-cfe5-425d-9571-742c1956175c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.341450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 935/1883] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:18.346116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.357334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de631c7-9a7f-4fd5-90fd-7bbd41367c36 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.363145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.363851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.364137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.364412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] PATH_INFO: `/v3/projects/7e0b8d362c09418fa7713a140dee7095/users/54fcc77e2cfe4c7d801b76ea0c563ae6/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.381333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0b8d362c09418fa7713a140dee7095, user_id=54fcc77e2cfe4c7d801b76ea0c563ae6, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.384788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.400553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 949/1884] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:18 2022] PUT /identity/v3/projects/7e0b8d362c09418fa7713a140dee7095/users/54fcc77e2cfe4c7d801b76ea0c563ae6/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:18.411980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.412270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.412558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.447919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] MFA Rules not processed for user `54fcc77e2cfe4c7d801b76ea0c563ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:18.450104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.451819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:18.452247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:18.478255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:18.522886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 936/1885] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:18.534120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.535580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:18.537373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb88d-986b-4606-9130-6e3475063c66 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.543297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.544127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.544469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.544812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] PATH_INFO: `/v3/users/a04131c09c454436a61a1c0763a43dcc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.546005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04131c09c454436a61a1c0763a43dcc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.548294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.604871 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0679e-519d-4789-b06c-53f78bee267a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.609229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04131c09c454436a61a1c0763a43dcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:18.609857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f104907-1211-4af0-8cde-f36164c3a98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.610483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f104907-1211-4af0-8cde-f36164c3a98a service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.610768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f104907-1211-4af0-8cde-f36164c3a98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.611048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f104907-1211-4af0-8cde-f36164c3a98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.618459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04131c09c454436a61a1c0763a43dcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:18.623002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f104907-1211-4af0-8cde-f36164c3a98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.624492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f104907-1211-4af0-8cde-f36164c3a98a service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.632739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04131c09c454436a61a1c0763a43dcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:18.634808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04131c09c454436a61a1c0763a43dcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:18.649713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 937/1886] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:18.673123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04131c09c454436a61a1c0763a43dcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:18.686185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 950/1887] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:18 2022] DELETE /identity/v3/users/a04131c09c454436a61a1c0763a43dcc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:18.690570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.691043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.691451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.740692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:18.741923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:18.826118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 938/1888] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:18.836670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:18.838360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:18.916749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:18.946358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338adc6c-6eab-4f33-a067-c2126415421c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:18.951869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:18.953527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:18.953957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:18.954472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:18.968881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:18.974326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:18.987134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 951/1889] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:19.469844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:19.471588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:19.485475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3a59-c127-4010-911d-2ee91df9cbab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:19.490364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:19.491006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:19.491339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:19.491650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] PATH_INFO: `/v3/projects/a223481a0c3343e8a5c97f7bd7a2d68b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:19.493738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a223481a0c3343e8a5c97f7bd7a2d68b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:19.495804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:19.550601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a223481a0c3343e8a5c97f7bd7a2d68b', 'request_id': 'req-cee88260-45c2-42f2-a7e6-8b676608336a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.557520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a223481a0c3343e8a5c97f7bd7a2d68b', 'request_id': 'req-cee88260-45c2-42f2-a7e6-8b676608336a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.564494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a223481a0c3343e8a5c97f7bd7a2d68b', 'request_id': 'req-cee88260-45c2-42f2-a7e6-8b676608336a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.566290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:19.582238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] The token cache is being invalidate because project a223481a0c3343e8a5c97f7bd7a2d68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:19.582534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.583736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.585522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 939/1890] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:19 2022] DELETE /identity/v3/projects/a223481a0c3343e8a5c97f7bd7a2d68b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:19.590447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:19.601333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698ef2d-4fe9-4c1a-bd81-02330a760efc service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:19.664865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:19.665493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:19.665770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:19.666068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] PATH_INFO: `/v3/users/6f003a12d8ca405ba5465ce9bdbf9d91` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:19.667208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f003a12d8ca405ba5465ce9bdbf9d91)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:19.669146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:19.716929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f003a12d8ca405ba5465ce9bdbf9d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.724804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f003a12d8ca405ba5465ce9bdbf9d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.732038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f003a12d8ca405ba5465ce9bdbf9d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.732673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f003a12d8ca405ba5465ce9bdbf9d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.745681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0c68c19-34af-4c4b-86f3-08035d13d79b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f003a12d8ca405ba5465ce9bdbf9d91'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:19.751866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 952/1891] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:19 2022] DELETE /identity/v3/users/6f003a12d8ca405ba5465ce9bdbf9d91 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:20.028129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.029671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:20.043282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee88260-45c2-42f2-a7e6-8b676608336a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.120514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.121154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.121434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.121710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] PATH_INFO: `/v3/projects/00386e4803454167b4c814b54100c20a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.122827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00386e4803454167b4c814b54100c20a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.124938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.171586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00386e4803454167b4c814b54100c20a', 'request_id': 'req-79ba96f5-656f-4715-beeb-8f06449bb749'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:20.177107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00386e4803454167b4c814b54100c20a', 'request_id': 'req-79ba96f5-656f-4715-beeb-8f06449bb749'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:20.183365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00386e4803454167b4c814b54100c20a', 'request_id': 'req-79ba96f5-656f-4715-beeb-8f06449bb749'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:20.196376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] The token cache is being invalidate because project 00386e4803454167b4c814b54100c20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:20.196667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:20.197295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79ba96f5-656f-4715-beeb-8f06449bb749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:20.198792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 940/1892] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:20 2022] DELETE /identity/v3/projects/00386e4803454167b4c814b54100c20a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:20.203865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.204159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.204435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.226053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:20.227305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:20.292445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 953/1893] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:20.300260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.300566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.301020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.322428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:20.323676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:20.331543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 941/1894] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:20.335997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.341745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:20.348406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ee402-35d7-4a29-b388-f468709f2ef1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.354275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.354895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.355175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.355465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.356132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:20.356549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.358148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.370674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 954/1895] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:20.374909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.385147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e36c9-4b02-44bd-baae-b2d49d047412 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.389974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.390650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.390962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.391263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.391921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.393822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.422971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 942/1896] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:20.427014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.438994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da235-07dd-4ad3-84ff-5e1c462ca30d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.495277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.495975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.496379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.496655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.497347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.499261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.523889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 955/1897] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:20.525173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.538677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fc4df-69a7-473c-aa7d-755f4b66493d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.543883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.544488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.544763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.545033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.545763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.547351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.558535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 943/1898] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:20.562744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.573703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb63ea-d97b-443a-984d-c77e7be439d3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.578580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae0422-d553-4c3e-850f-0a0904c1764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.579189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0422-d553-4c3e-850f-0a0904c1764b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.579471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0422-d553-4c3e-850f-0a0904c1764b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.579785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0422-d553-4c3e-850f-0a0904c1764b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.580436 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae0422-d553-4c3e-850f-0a0904c1764b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.582042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae0422-d553-4c3e-850f-0a0904c1764b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.593671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 956/1899] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:20.597922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.608955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668888a4-7522-4f4f-b778-977b1254e3f7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.613610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.614263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.614546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.614824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] PATH_INFO: `/v3/projects/3d3d1a4a472b48ebb9f7586f2368f4b1/users/3e457583a7a146bcb9431a74027e4db1/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.631994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3d1a4a472b48ebb9f7586f2368f4b1, user_id=3e457583a7a146bcb9431a74027e4db1, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.635296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.655700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 944/1900] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:20 2022] PUT /identity/v3/projects/3d3d1a4a472b48ebb9f7586f2368f4b1/users/3e457583a7a146bcb9431a74027e4db1/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:20.662907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.663498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.665156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.697493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] MFA Rules not processed for user `3e457583a7a146bcb9431a74027e4db1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:20.698761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:20.768384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 957/1901] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:20.772809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.793159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:20.844243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083f2f4-0e65-4289-9571-509dece67d56 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.848961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.849571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.849855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.850133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.850761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.852753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.880408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 945/1902] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:20.885120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:20.895886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ccf26-a918-431a-857a-ce109d02028b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:20.948792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:20.956260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:20.957050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:20.957613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:20.958050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:20.958684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:20.960670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:20.987755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 958/1903] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:20 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:20.991983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:21.003175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465323a-5e8d-4b78-9864-214854a0f1e9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:21.008800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:21.009492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:21.009801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:21.010053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:21.010701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:21.012337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:21.027619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 946/1904] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:21.032798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:21.046750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00759326-71a2-4d5d-b2dd-5b69dbe4fe72 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:21.053264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:21.054263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:21.054395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:21.054681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] PATH_INFO: `/v3/projects/d1149c6a0f1c4f89863a515c7fb078d4/users/2cfca89458de4a618a4672ec73402a64/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:21.083618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1149c6a0f1c4f89863a515c7fb078d4, user_id=2cfca89458de4a618a4672ec73402a64, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:21.087165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:21.102569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 959/1905] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:21 2022] PUT /identity/v3/projects/d1149c6a0f1c4f89863a515c7fb078d4/users/2cfca89458de4a618a4672ec73402a64/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:21.106911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:21.119717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616894-c1af-44ee-86dc-d989b6fcecc1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:21.252142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e0ee0-6598-4616-87c9-d44836d4f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:21.252843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0ee0-6598-4616-87c9-d44836d4f398 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:21.253120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0ee0-6598-4616-87c9-d44836d4f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:21.253496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0ee0-6598-4616-87c9-d44836d4f398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:21.254135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e0ee0-6598-4616-87c9-d44836d4f398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:21.256158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e0ee0-6598-4616-87c9-d44836d4f398 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:21.267881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 947/1906] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:21.271923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:21.283241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5342b8-7fe5-4360-a380-c51715a4117d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:21.288037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:21.288731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:21.289020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:21.289327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] PATH_INFO: `/v3/projects/d1149c6a0f1c4f89863a515c7fb078d4/users/2cfca89458de4a618a4672ec73402a64/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:21.291705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1149c6a0f1c4f89863a515c7fb078d4, user_id=2cfca89458de4a618a4672ec73402a64, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:21.294527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:21.308895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 960/1907] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:21 2022] PUT /identity/v3/projects/d1149c6a0f1c4f89863a515c7fb078d4/users/2cfca89458de4a618a4672ec73402a64/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:21.319332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:21.319646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:21.319933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:21.352565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] MFA Rules not processed for user `2cfca89458de4a618a4672ec73402a64`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:21.353952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:21.432833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 948/1908] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:21.443680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:21.445472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:21.514054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5da46-ca38-4f10-8dd5-aaa05ef62fda admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:21.518647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb279f-aedd-4522-8c74-b4a86c1955d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:21.519268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb279f-aedd-4522-8c74-b4a86c1955d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:21.519552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb279f-aedd-4522-8c74-b4a86c1955d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:21.522143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb279f-aedd-4522-8c74-b4a86c1955d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:21.531929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb279f-aedd-4522-8c74-b4a86c1955d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:21.533419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb279f-aedd-4522-8c74-b4a86c1955d1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:21.545439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 961/1909] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:22.204520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:22.219330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:22.273416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd41262-55be-4908-a981-b074bb6f96ac None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:22.277998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:22.278631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:22.278912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:22.279182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] PATH_INFO: `/v3/users/3e457583a7a146bcb9431a74027e4db1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:22.280474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e457583a7a146bcb9431a74027e4db1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:22.282373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:22.331339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e457583a7a146bcb9431a74027e4db1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.338461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e457583a7a146bcb9431a74027e4db1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.339151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e457583a7a146bcb9431a74027e4db1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.345816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e457583a7a146bcb9431a74027e4db1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.358249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e457583a7a146bcb9431a74027e4db1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.364248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 949/1910] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:22 2022] DELETE /identity/v3/users/3e457583a7a146bcb9431a74027e4db1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:22.369461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:22.369933 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:22.370027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:22.389055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:22.390274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:22.447527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 962/1911] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:22.457181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:22.458633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:22.516212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26b441-6d29-4ba9-bd59-08173f8c3180 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:22.519919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:22.520524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:22.522078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:22.522078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:22.531854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:22.533356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:22.543618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 950/1912] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:22.865320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:22.866889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:22.884938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c102b2-4098-49f5-be66-ed90a0aa35a4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:22.889778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:22.890500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:22.890825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:22.891040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] PATH_INFO: `/v3/projects/3d3d1a4a472b48ebb9f7586f2368f4b1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:22.892193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3d1a4a472b48ebb9f7586f2368f4b1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:22.894239 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:22.945295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:22.947009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3d1a4a472b48ebb9f7586f2368f4b1', 'request_id': 'req-7563d2b5-2e10-456e-a2e0-984f7a360fab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.955924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3d1a4a472b48ebb9f7586f2368f4b1', 'request_id': 'req-7563d2b5-2e10-456e-a2e0-984f7a360fab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.963894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3d1a4a472b48ebb9f7586f2368f4b1', 'request_id': 'req-7563d2b5-2e10-456e-a2e0-984f7a360fab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.974648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] The token cache is being invalidate because project 3d3d1a4a472b48ebb9f7586f2368f4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:22.974971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.975575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:22.977800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 963/1913] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:22 2022] DELETE /identity/v3/projects/3d3d1a4a472b48ebb9f7586f2368f4b1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:22.981913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:22.991367 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a2f13-a9b4-48dd-9776-4b04c7c31ea2 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:23.024672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:23.050879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:23.051508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:23.051824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:23.052098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] PATH_INFO: `/v3/users/2cfca89458de4a618a4672ec73402a64` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:23.053205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cfca89458de4a618a4672ec73402a64)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:23.055149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:23.101005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cfca89458de4a618a4672ec73402a64'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.112176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cfca89458de4a618a4672ec73402a64'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.113142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cfca89458de4a618a4672ec73402a64'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.122122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cfca89458de4a618a4672ec73402a64'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.139207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c9fba662-9629-4539-ae80-fddfb8618d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cfca89458de4a618a4672ec73402a64'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.145862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 951/1914] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:22 2022] DELETE /identity/v3/users/2cfca89458de4a618a4672ec73402a64 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:23.631847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:23.633960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:23.650707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563d2b5-2e10-456e-a2e0-984f7a360fab admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:23.707631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:23.708272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:23.708555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:23.708827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] PATH_INFO: `/v3/projects/d1149c6a0f1c4f89863a515c7fb078d4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:23.709938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1149c6a0f1c4f89863a515c7fb078d4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:23.712019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:23.759207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1149c6a0f1c4f89863a515c7fb078d4', 'request_id': 'req-6056433b-b2c9-4cbb-b0af-0426aef54e04'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.766146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1149c6a0f1c4f89863a515c7fb078d4', 'request_id': 'req-6056433b-b2c9-4cbb-b0af-0426aef54e04'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.772329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1149c6a0f1c4f89863a515c7fb078d4', 'request_id': 'req-6056433b-b2c9-4cbb-b0af-0426aef54e04'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.783076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] The token cache is being invalidate because project d1149c6a0f1c4f89863a515c7fb078d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:23.783366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.784027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6056433b-b2c9-4cbb-b0af-0426aef54e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:23.785534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 964/1915] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:23 2022] DELETE /identity/v3/projects/d1149c6a0f1c4f89863a515c7fb078d4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:23.791226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:23.791710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:23.791801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:23.811161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:23.812392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:23.875713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 952/1916] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:23.884376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:23.884667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:23.884948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:23.908699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:23.910472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:23.923534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 965/1917] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:23.927687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:23.943551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bbad3-56b7-4d76-a2d3-33da25f17038 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:23.951684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:23.953003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:23.953382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:23.953755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:23.954518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:23.954984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:23.957137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:23.975144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 953/1918] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:23.979815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:23.991119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dbc9a-8981-4f8e-8346-af138e191331 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:23.999201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:23.999989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.000367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.000738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.001468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.004061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.042619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 966/1919] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:23 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:24.046644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:24.057889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721d7b3-ce45-4a92-b68e-5921ee9d42b5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:24.138264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:24.154305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:24.155149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.155500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.155914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.156670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.159932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.188207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 954/1920] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:24 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:24.189487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:24.201350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d96d6-fb6c-4b8f-8b22-55d01b99bf07 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:24.206274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:24.208147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.208147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.208147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.209135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.210330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.239243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 967/1921] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:24.240734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:24.253843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b26f70-8880-4857-82de-10537cb92af5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:24.259628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07c952-225a-437a-9305-3585c53990ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:24.260369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07c952-225a-437a-9305-3585c53990ed admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.260793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07c952-225a-437a-9305-3585c53990ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.261170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07c952-225a-437a-9305-3585c53990ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.261948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07c952-225a-437a-9305-3585c53990ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.263712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07c952-225a-437a-9305-3585c53990ed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.279022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 955/1922] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:24.281221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:24.293549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4520-2f90-411f-9e3d-c31c39fb522b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:24.298558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:24.299344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.299485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.299790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] PATH_INFO: `/v3/projects/145b73ca2b5740a9a728f2f4f3a56a51/users/9f79b21824114a7e9bec28b0b4b0f3a2/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.323179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] RBAC: Authorizing `identity:create_grant(project_id=145b73ca2b5740a9a728f2f4f3a56a51, user_id=9f79b21824114a7e9bec28b0b4b0f3a2, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.326165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.343908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 968/1923] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:24 2022] PUT /identity/v3/projects/145b73ca2b5740a9a728f2f4f3a56a51/users/9f79b21824114a7e9bec28b0b4b0f3a2/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:24.355825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.356125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.356380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.388078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] MFA Rules not processed for user `9f79b21824114a7e9bec28b0b4b0f3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:24.389305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:24.497218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 956/1924] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:24.513016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:24.514909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:24.521853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:24.604949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c947df-cc11-4a13-bcda-74efd94180af admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:24.611787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbda124-1918-4a91-bd3f-4c8a68151407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:24.612683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbda124-1918-4a91-bd3f-4c8a68151407 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.613088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbda124-1918-4a91-bd3f-4c8a68151407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.613803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbda124-1918-4a91-bd3f-4c8a68151407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.625895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbda124-1918-4a91-bd3f-4c8a68151407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.627822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbda124-1918-4a91-bd3f-4c8a68151407 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.640692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 969/1925] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:24.662845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:24.780904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824231a-c316-4ba3-8d88-09900f30856e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:24.788627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:24.791187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.791476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.792130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] PATH_INFO: `/v3/users/9f79b21824114a7e9bec28b0b4b0f3a2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.793398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f79b21824114a7e9bec28b0b4b0f3a2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:24.795989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:24.856773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f79b21824114a7e9bec28b0b4b0f3a2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:24.863923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:24.867786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f79b21824114a7e9bec28b0b4b0f3a2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:24.868541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f79b21824114a7e9bec28b0b4b0f3a2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:24.881109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f79b21824114a7e9bec28b0b4b0f3a2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:24.897081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f79b21824114a7e9bec28b0b4b0f3a2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:24.904227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 957/1926] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:24 2022] DELETE /identity/v3/users/9f79b21824114a7e9bec28b0b4b0f3a2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:24.908919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:24.909255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:24.909510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:24.931688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:24.932904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:24.992745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 970/1927] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:25.002458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.003943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.067483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ef053-0bed-4615-8446-3d72b4c70587 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.071728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df18c9-b9d3-429e-89ce-daa6ed4a835a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.072341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df18c9-b9d3-429e-89ce-daa6ed4a835a service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.072616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df18c9-b9d3-429e-89ce-daa6ed4a835a service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.072890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df18c9-b9d3-429e-89ce-daa6ed4a835a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.083502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df18c9-b9d3-429e-89ce-daa6ed4a835a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.085226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df18c9-b9d3-429e-89ce-daa6ed4a835a service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.097998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 958/1928] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:25.105681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.107270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.119726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db610a-749b-4004-b3aa-ce8117fbc81d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.124784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.125579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.125920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.126126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] PATH_INFO: `/v3/users/54fcc77e2cfe4c7d801b76ea0c563ae6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.127268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fcc77e2cfe4c7d801b76ea0c563ae6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.129481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.187027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fcc77e2cfe4c7d801b76ea0c563ae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.193753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fcc77e2cfe4c7d801b76ea0c563ae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.201319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fcc77e2cfe4c7d801b76ea0c563ae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.202331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fcc77e2cfe4c7d801b76ea0c563ae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.215734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fcc77e2cfe4c7d801b76ea0c563ae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.222033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 971/1929] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:25 2022] DELETE /identity/v3/users/54fcc77e2cfe4c7d801b76ea0c563ae6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:25.227257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.227688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.228029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.247198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:25.248533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.312006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 959/1930] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:25.322370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.324298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.345642 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:25.392717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.394939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.399515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa1448-c0b2-4211-8d09-26efd5694f90 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.403478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09814aaa-e82e-4406-a43f-5f855977d7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.404268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09814aaa-e82e-4406-a43f-5f855977d7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.404643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09814aaa-e82e-4406-a43f-5f855977d7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.405068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09814aaa-e82e-4406-a43f-5f855977d7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.416455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09814aaa-e82e-4406-a43f-5f855977d7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.418000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09814aaa-e82e-4406-a43f-5f855977d7c0 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.424994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2bed4-c15e-4f8d-889e-2e3044a58138 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.429181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 972/1931] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:25.431294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.432037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.432329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.432595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] PATH_INFO: `/v3/projects/145b73ca2b5740a9a728f2f4f3a56a51` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.433937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145b73ca2b5740a9a728f2f4f3a56a51)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.435884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.498478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145b73ca2b5740a9a728f2f4f3a56a51', 'request_id': 'req-50b94ccd-2a85-4d74-833f-04616d627990'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.508157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145b73ca2b5740a9a728f2f4f3a56a51', 'request_id': 'req-50b94ccd-2a85-4d74-833f-04616d627990'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.516283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145b73ca2b5740a9a728f2f4f3a56a51', 'request_id': 'req-50b94ccd-2a85-4d74-833f-04616d627990'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.522952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:25.533648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] The token cache is being invalidate because project 145b73ca2b5740a9a728f2f4f3a56a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:25.534260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.535519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50b94ccd-2a85-4d74-833f-04616d627990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:25.541414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 960/1932] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:25 2022] DELETE /identity/v3/projects/145b73ca2b5740a9a728f2f4f3a56a51 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:25.543663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.544176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.544470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.565581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:25.566817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.634644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 973/1933] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:25.642571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.642874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.643270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.666104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:25.667224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.675071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 961/1934] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:25.680131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.692358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30ee96-0949-4f3d-983c-bed8b11f86f4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.697965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.698576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.698851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.699124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.699800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:25.700102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.701694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.714563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 974/1935] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:25.719184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.728701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d6331-66ef-4fc4-b4a0-25459ed514bd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.733511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.734121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.734396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.734664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.735362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.737388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.767719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 962/1936] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:25 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:25.771967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.787789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4fa5b-90bd-4402-ad54-1be748656374 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.800113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:25.801942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:25.886260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649437-b397-4598-8772-4ec6cdac06ec admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:25.907362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.909221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.909221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.910438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:25.911097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:25.911522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:25.911885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] PATH_INFO: `/v3/projects/7e0b8d362c09418fa7713a140dee7095` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.912459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:25.913133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.915696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0b8d362c09418fa7713a140dee7095)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:25.918214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.919926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:25.972188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 975/1937] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:25 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:25.974040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.012690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedc4fd-fe60-4e7c-b92e-f34d88c85675 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.013787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0b8d362c09418fa7713a140dee7095', 'request_id': 'req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:26.032482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0b8d362c09418fa7713a140dee7095', 'request_id': 'req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:26.052314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0b8d362c09418fa7713a140dee7095', 'request_id': 'req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:26.073824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] The token cache is being invalidate because project 7e0b8d362c09418fa7713a140dee7095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:26.075269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:26.077120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4d7f2d23-28ba-432c-a771-e1d5c9fb703f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:26.079253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 963/1938] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:25 2022] DELETE /identity/v3/projects/7e0b8d362c09418fa7713a140dee7095 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:26.087333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.087947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.088370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.126348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:26.136203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:26.137466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:26.200825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.201556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.201912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.202259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.202997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:26.204824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:26.223354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 976/1939] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:26.227733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.235852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:26.274598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16c65b-b90c-4bab-9459-f0ce0c0e1ac5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.287464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e659cc-958a-49a9-bb3d-e6f1bfdec24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.288318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e659cc-958a-49a9-bb3d-e6f1bfdec24e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.294149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e659cc-958a-49a9-bb3d-e6f1bfdec24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.294440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e659cc-958a-49a9-bb3d-e6f1bfdec24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.295171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e659cc-958a-49a9-bb3d-e6f1bfdec24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:26.300042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e659cc-958a-49a9-bb3d-e6f1bfdec24e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:26.300747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:26.322702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 964/1940] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:26.323349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 977/1941] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:26.325250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.329676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.330817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.331288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.348079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023d8ae-9bce-4d74-bc60-cc3c2920e18b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.366584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a729f-850c-48b9-8639-42bc331d4506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.374740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a729f-850c-48b9-8639-42bc331d4506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.375034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a729f-850c-48b9-8639-42bc331d4506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.375320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a729f-850c-48b9-8639-42bc331d4506 admin admin] PATH_INFO: `/v3/projects/7f0f51c6328d4b37b196d0c17e6abed7/users/f5e97ea3f6e6469aae5451c4aa391699/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.378716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:26.380753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:26.414368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a729f-850c-48b9-8639-42bc331d4506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0f51c6328d4b37b196d0c17e6abed7, user_id=f5e97ea3f6e6469aae5451c4aa391699, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:26.414709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 978/1942] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:26.415910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.417356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a729f-850c-48b9-8639-42bc331d4506 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:26.434160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5994e9e-9bd3-4528-8434-7c836e25515c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.452019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.452772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.453146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.453515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.454270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:26.454674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:26.455939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 965/1943] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:26 2022] PUT /identity/v3/projects/7f0f51c6328d4b37b196d0c17e6abed7/users/f5e97ea3f6e6469aae5451c4aa391699/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:26.457504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:26.468970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.469755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.470335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.497835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 979/1944] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:03:26.502066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.515388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb702cf-4159-44d8-b76e-ccabab6cf5f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.540065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] MFA Rules not processed for user `f5e97ea3f6e6469aae5451c4aa391699`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:26.541645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:26.626697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.628034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.628317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.630798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.631561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:26.637852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:26.684422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 966/1945] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:26.700180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.703069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:26.713454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 980/1946] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:26 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:26.717541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.917199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6971-9558-4146-a3df-5cbd19d03323 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.918868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34991e-b036-4711-aafe-d518d34089fa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:26.931370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.934812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.935034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.935313 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.937028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:26.941303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:26.979567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:26.980427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:26.980706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:26.980976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:26.987321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 981/1947] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:26 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:26.992792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:26.997480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:27.010277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eba76-d5c3-4d05-89bc-b4c2ab3fa844 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:27.021101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:27.021846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:27.022130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:27.022407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:27.023153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:27.024930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:27.045010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 982/1948] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:27.048261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:27.055769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:27.057141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:27.068713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 967/1949] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:27.073335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f16a3e-7e3d-4bf2-bc2d-46aaea849a66 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:27.083345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17cc63-bb1c-4c22-8649-70577b5a4c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:27.084095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cc63-bb1c-4c22-8649-70577b5a4c10 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:27.084607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cc63-bb1c-4c22-8649-70577b5a4c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:27.084777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cc63-bb1c-4c22-8649-70577b5a4c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:27.085450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17cc63-bb1c-4c22-8649-70577b5a4c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:27.087848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17cc63-bb1c-4c22-8649-70577b5a4c10 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:27.089397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:27.112638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 983/1950] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:27.114955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:27.129778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47073011-94c9-492b-a8d0-c29d56d14267 service glance] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:27.136889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:27.137552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:27.137887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:27.138169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] PATH_INFO: `/v3/projects/814c528dd56c4e61ae5cba5733bf2f08/users/2a6b55d7cb1840a5ad93369589a36596/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:27.160839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=814c528dd56c4e61ae5cba5733bf2f08, user_id=2a6b55d7cb1840a5ad93369589a36596, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:27.167414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:27.191652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 968/1951] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:27 2022] PUT /identity/v3/projects/814c528dd56c4e61ae5cba5733bf2f08/users/2a6b55d7cb1840a5ad93369589a36596/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:27.194466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef79799-5a78-4f30-bb83-c8f2d2b2c09c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:27.194844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef79799-5a78-4f30-bb83-c8f2d2b2c09c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:27.196028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef79799-5a78-4f30-bb83-c8f2d2b2c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:27.216912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8ef79799-5a78-4f30-bb83-c8f2d2b2c09c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:27.218397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ef79799-5a78-4f30-bb83-c8f2d2b2c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:27.307366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 984/1952] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:27.326007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:27.328416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:27.407682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06696561-0457-4a26-9fcd-3fc53b2c3a8d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:27.411948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:27.412510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:27.412790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:27.413062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:27.426401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:27.428151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:27.441939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 969/1953] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:28.683054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa4892-43a7-4b13-8c65-dee5608cf6a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:28.683485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa4892-43a7-4b13-8c65-dee5608cf6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:28.683485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa4892-43a7-4b13-8c65-dee5608cf6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:28.752737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1faa4892-43a7-4b13-8c65-dee5608cf6a2 None None] MFA Rules not processed for user `2a6b55d7cb1840a5ad93369589a36596`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:28.754169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1faa4892-43a7-4b13-8c65-dee5608cf6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:28.946429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 985/1954] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:28.958229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:28.963566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:29.072145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718aa61-802d-4bc6-9f7c-728fdb9ad794 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:29.082133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:29.082978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:29.083305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:29.083533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:29.103728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:29.105069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:29.119943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 970/1955] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:30.605837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0f3b7-f607-49cb-8429-1a869d615305 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:30.606367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0f3b7-f607-49cb-8429-1a869d615305 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:30.606483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0f3b7-f607-49cb-8429-1a869d615305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:30.639497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a6a0f3b7-f607-49cb-8429-1a869d615305 None None] MFA Rules not processed for user `2a6b55d7cb1840a5ad93369589a36596`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:30.641130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6a0f3b7-f607-49cb-8429-1a869d615305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:30.651441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 986/1956] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:38.143847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:38.145303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:38.159045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aca19f-16be-4ecf-a758-b29bfcfbcbf1 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:38.163754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:38.164399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:38.165348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:38.165629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] PATH_INFO: `/v3/users/03fe851af95548e6bd74b1942fe581d7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:38.166770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03fe851af95548e6bd74b1942fe581d7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:38.169076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:38.208023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:38.224540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03fe851af95548e6bd74b1942fe581d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:38.234457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03fe851af95548e6bd74b1942fe581d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:38.235169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03fe851af95548e6bd74b1942fe581d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:38.244694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03fe851af95548e6bd74b1942fe581d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:38.257507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03fe851af95548e6bd74b1942fe581d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:38.263884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 971/1957] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:38 2022] DELETE /identity/v3/users/03fe851af95548e6bd74b1942fe581d7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:38.268551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:38.269024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:38.269227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:38.313877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:38.315370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:38.440080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 987/1958] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:38.452850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:38.454700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:38.529348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d3f21-8301-4a4b-8d09-c6ac9d2a11a3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:38.532939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:38.533546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:38.533824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:38.534096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:38.543767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:38.545213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:38.555249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 972/1959] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:39.237421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:39.238872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:39.252403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be63c1a-971c-4ef0-ba83-7eed3f70bbaf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:39.259262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:39.259500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:39.259974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:39.260249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] PATH_INFO: `/v3/projects/c37d5a3b88ab4b169e3f3e4e423a4498` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:39.261588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37d5a3b88ab4b169e3f3e4e423a4498)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:39.263491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:39.296346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:39.338989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37d5a3b88ab4b169e3f3e4e423a4498', 'request_id': 'req-37854403-f457-4809-8a7a-f4c27056fa53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.348925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37d5a3b88ab4b169e3f3e4e423a4498', 'request_id': 'req-37854403-f457-4809-8a7a-f4c27056fa53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.361748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37d5a3b88ab4b169e3f3e4e423a4498', 'request_id': 'req-37854403-f457-4809-8a7a-f4c27056fa53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.377008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] The token cache is being invalidate because project c37d5a3b88ab4b169e3f3e4e423a4498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:39.377299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.378187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.380014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 988/1960] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:39 2022] DELETE /identity/v3/projects/c37d5a3b88ab4b169e3f3e4e423a4498 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:39.384504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:39.399071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200bdef-42aa-4027-9a6c-68cdda7719af service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:39.510784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:39.511591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:39.511903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:39.512185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] PATH_INFO: `/v3/users/500bb289055d4f41af0dfb01c121ae9e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:39.517678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500bb289055d4f41af0dfb01c121ae9e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:39.519651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:39.612268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500bb289055d4f41af0dfb01c121ae9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.619104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:39.623314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500bb289055d4f41af0dfb01c121ae9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.625149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500bb289055d4f41af0dfb01c121ae9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.637141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500bb289055d4f41af0dfb01c121ae9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.668973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d645b967-07b8-43c4-92a9-87548cb19a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500bb289055d4f41af0dfb01c121ae9e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:39.679551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 973/1961] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:39 2022] DELETE /identity/v3/users/500bb289055d4f41af0dfb01c121ae9e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:40.585146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:40.587844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:40.605623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37854403-f457-4809-8a7a-f4c27056fa53 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:40.681353 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:40.682000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:40.682285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:40.682571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] PATH_INFO: `/v3/projects/4cae8fad268e4a69acfa71713aaca9a3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:40.683905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cae8fad268e4a69acfa71713aaca9a3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:40.685837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:40.736796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cae8fad268e4a69acfa71713aaca9a3', 'request_id': 'req-90eee61f-fb93-40cf-a88d-eafc4f25283e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:40.745464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cae8fad268e4a69acfa71713aaca9a3', 'request_id': 'req-90eee61f-fb93-40cf-a88d-eafc4f25283e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:40.752365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cae8fad268e4a69acfa71713aaca9a3', 'request_id': 'req-90eee61f-fb93-40cf-a88d-eafc4f25283e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:40.765316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] The token cache is being invalidate because project 4cae8fad268e4a69acfa71713aaca9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:40.765610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:40.766278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90eee61f-fb93-40cf-a88d-eafc4f25283e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:40.768923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 989/1962] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:40 2022] DELETE /identity/v3/projects/4cae8fad268e4a69acfa71713aaca9a3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:40.773866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:40.774432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:40.774618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:40.794914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:40.796083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:40.861494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 974/1963] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:40.870197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:40.870670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:40.871016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:40.892546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:40.893796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:40.902619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 990/1964] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:40.907003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:40.917126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed029c4-5b8d-4212-b4c8-a50fe0462943 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:40.921733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:40.922365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:40.922643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:40.922912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:40.923555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:40.923892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:40.925483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:40.945398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 975/1965] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:40.947190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:40.957586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b8c7-c8f1-44ee-a773-41d4fa67bda5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:40.962596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:40.963223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:40.963490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:40.963793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:40.964435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:40.966350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:40.995056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 991/1966] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:40 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:40.999504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.010394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc062d7-824e-4ea7-9d6b-d6435834455b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.077327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.077950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.078257 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.078531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.079158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.081109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.105129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 976/1967] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:40 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:41.109463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.118589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:41.125741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a578e-ab7d-4fdb-a179-778eb7b26908 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.131158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.132050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.132328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.132599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.133295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.135672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.149821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 992/1968] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:41.151301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.163519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b148-c05b-4dcf-806e-a637fed9b211 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.168427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1e4da-10a5-437e-85fe-a7337e2185e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.169029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e4da-10a5-437e-85fe-a7337e2185e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.169305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e4da-10a5-437e-85fe-a7337e2185e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.169573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e4da-10a5-437e-85fe-a7337e2185e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.170217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1e4da-10a5-437e-85fe-a7337e2185e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.171861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1e4da-10a5-437e-85fe-a7337e2185e4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.183729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 977/1969] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:41.187879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.198989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b76f6-c06a-4937-b214-f2674d32f620 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.203777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.204671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.205014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.205355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] PATH_INFO: `/v3/projects/646ce05103f943c4943133590ccf783b/users/16b01171dff342bfb599ded091a78b86/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.221168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ce05103f943c4943133590ccf783b, user_id=16b01171dff342bfb599ded091a78b86, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.223996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.237681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 993/1970] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:41 2022] PUT /identity/v3/projects/646ce05103f943c4943133590ccf783b/users/16b01171dff342bfb599ded091a78b86/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:41.250299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.250687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.251037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.281753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] MFA Rules not processed for user `16b01171dff342bfb599ded091a78b86`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:41.282997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:41.309367 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:41.355551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 978/1971] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:41.360124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.425836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bb67f-ab21-4453-a6ba-6f60ee4caafe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.431663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.432293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.432579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.432858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.433617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.435561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.469722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 994/1972] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:41 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:41.473895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.486159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c24740-fc74-4fd0-a747-e109fedde7e1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.552546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.553150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.553426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.553707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.554337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.556919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.604182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 979/1973] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:41 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:41.607624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.623539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a33c6-bb92-43af-b86d-882747bd8a34 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.630555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.631452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.631930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.632361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.633330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.644587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.659647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 995/1974] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:41.669289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.684296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db8029-90b2-43b9-afde-4c7ff5ffe1a1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.693855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.695080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.695080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.695080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] PATH_INFO: `/v3/projects/595ef7ec4f1742c195d4f9bdfe615a0d/users/dd4c69e3e0714a67908bb4de65b42d89/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.742541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=595ef7ec4f1742c195d4f9bdfe615a0d, user_id=dd4c69e3e0714a67908bb4de65b42d89, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.746493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.764622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 980/1975] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:41 2022] PUT /identity/v3/projects/595ef7ec4f1742c195d4f9bdfe615a0d/users/dd4c69e3e0714a67908bb4de65b42d89/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:41.768735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.779459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6b7ee-9cfb-4d50-adef-7183d401cda9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.853173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594e31c-c8f9-46be-8d30-ff8f11665814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.853859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594e31c-c8f9-46be-8d30-ff8f11665814 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.854144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594e31c-c8f9-46be-8d30-ff8f11665814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.854496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594e31c-c8f9-46be-8d30-ff8f11665814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.860371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594e31c-c8f9-46be-8d30-ff8f11665814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.862089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594e31c-c8f9-46be-8d30-ff8f11665814 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.873566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 996/1976] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:03:41.877955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:41.893514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3e65c-7399-4851-a564-92818a594e74 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:41.898306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:41.898973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.899320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.899624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] PATH_INFO: `/v3/projects/595ef7ec4f1742c195d4f9bdfe615a0d/users/dd4c69e3e0714a67908bb4de65b42d89/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.901611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=595ef7ec4f1742c195d4f9bdfe615a0d, user_id=dd4c69e3e0714a67908bb4de65b42d89, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:41.904295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:41.915624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 981/1977] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:41 2022] PUT /identity/v3/projects/595ef7ec4f1742c195d4f9bdfe615a0d/users/dd4c69e3e0714a67908bb4de65b42d89/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:41.926832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:41.927132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:41.927620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:41.959632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] MFA Rules not processed for user `dd4c69e3e0714a67908bb4de65b42d89`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:41.961084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:41.996549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:42.046733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 997/1978] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:42.054460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:42.056808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:42.118475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eddea-9e21-4e86-a543-154d31a2a18f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:42.122537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:42.123147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:42.123427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:42.123723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:42.159480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:42.189439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:42.190891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:42.200531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 982/1979] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:47.995700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:48.005631 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9ebb-7c8a-4604-8879-b8a0ade0c4a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:48.010033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e859c-e672-40f0-99b4-f156ab076859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:48.010718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e859c-e672-40f0-99b4-f156ab076859 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:48.011065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e859c-e672-40f0-99b4-f156ab076859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:48.011776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e859c-e672-40f0-99b4-f156ab076859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:48.021689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e859c-e672-40f0-99b4-f156ab076859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:48.023199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e859c-e672-40f0-99b4-f156ab076859 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:48.034666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 998/1980] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:52.196510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=13 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:54.709961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:54.711436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:54.778188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf425f-3e2e-4108-9663-85fe863993f6 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:54.783480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:54.784153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:54.784435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:54.784713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] PATH_INFO: `/v3/users/f5e97ea3f6e6469aae5451c4aa391699` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:54.785942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5e97ea3f6e6469aae5451c4aa391699)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:54.787913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:54.835185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5e97ea3f6e6469aae5451c4aa391699'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:54.847788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5e97ea3f6e6469aae5451c4aa391699'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:54.848581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5e97ea3f6e6469aae5451c4aa391699'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:54.857545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5e97ea3f6e6469aae5451c4aa391699'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:54.872603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5e97ea3f6e6469aae5451c4aa391699'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:54.878501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 983/1981] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:54 2022] DELETE /identity/v3/users/f5e97ea3f6e6469aae5451c4aa391699 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:54.883405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:54.883891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:54.884022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:54.903322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:54.904569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:54.965870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 999/1982] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:54.975573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:54.977671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:55.039903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fdc4-a9c0-4ad2-a87c-51117ffd8edd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:55.043436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc55b84-63b1-4d68-abb1-c09a93c7ecd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:55.044081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55b84-63b1-4d68-abb1-c09a93c7ecd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:55.044364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55b84-63b1-4d68-abb1-c09a93c7ecd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:55.044644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55b84-63b1-4d68-abb1-c09a93c7ecd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:55.053615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc55b84-63b1-4d68-abb1-c09a93c7ecd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:55.055141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc55b84-63b1-4d68-abb1-c09a93c7ecd2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:55.065455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 984/1983] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:55.708601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:55.710327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:55.723570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3daa3b-1874-48bc-aa84-8fcb6bf0fda5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:55.736358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:55.737080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:55.737396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:55.737705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] PATH_INFO: `/v3/projects/7f0f51c6328d4b37b196d0c17e6abed7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:55.739273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0f51c6328d4b37b196d0c17e6abed7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:55.749085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:55.799698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0f51c6328d4b37b196d0c17e6abed7', 'request_id': 'req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:55.808476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0f51c6328d4b37b196d0c17e6abed7', 'request_id': 'req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:55.815224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0f51c6328d4b37b196d0c17e6abed7', 'request_id': 'req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:55.827107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] The token cache is being invalidate because project 7f0f51c6328d4b37b196d0c17e6abed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:55.827421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:55.828178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35fa9abb-cfe8-4397-9bbb-3ea17ca1ed34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:55.829731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1000/1984] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:55 2022] DELETE /identity/v3/projects/7f0f51c6328d4b37b196d0c17e6abed7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:55.835213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:55.835696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:55.835696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:55.861414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:55.863009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:55.882480 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:55.932571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 985/1985] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:55.941034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:55.941571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:55.941571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:55.963318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:55.964569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:55.972879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1001/1986] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:55.977713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:55.987919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19451e61-04e7-468e-9ebb-0f8c41e63ec3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:55.993836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:55.994449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:55.994723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:55.994992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:55.995676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:55.996107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:55.997700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.016072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 986/1987] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:56.017660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:56.027379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32b001-3193-42a6-add9-9861bd56aa83 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:56.032201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:56.032886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.033233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.033712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.034562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:56.036586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.080769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1002/1988] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:56 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:56.087072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:56.098133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9715e7c-6bfd-48eb-8faa-249b6a941b08 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:56.175794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:56.176498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.176841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.177222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.178929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:56.180984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.207797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 987/1989] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:56 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:03:56.209369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:56.220180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6971390-12c9-4636-a543-73a23150061a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:56.224989 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:56.225602 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.225882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.226154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.227462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:56.229025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.230437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:56.247061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:56.252713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1003/1990] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:56.259721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90410b74-39f7-4956-9479-c4cb02a65356 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:56.265683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9b48-51ba-4f9e-acce-5fedfaf8cd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:56.266303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9b48-51ba-4f9e-acce-5fedfaf8cd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.266582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9b48-51ba-4f9e-acce-5fedfaf8cd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.266852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9b48-51ba-4f9e-acce-5fedfaf8cd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.267501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f9b48-51ba-4f9e-acce-5fedfaf8cd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:56.269217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f9b48-51ba-4f9e-acce-5fedfaf8cd84 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.284883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 988/1991] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:56.286522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:56.298981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89debc71-83bb-4e9f-85df-883d9628007f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:56.303922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:56.305441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.306106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.306369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] PATH_INFO: `/v3/projects/8c78ed6f8b354b1b9b07151736df1386/users/3e146e6da0294fa78e07d8c1e971af3e/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.322486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c78ed6f8b354b1b9b07151736df1386, user_id=3e146e6da0294fa78e07d8c1e971af3e, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:56.325304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.340097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1004/1992] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:56 2022] PUT /identity/v3/projects/8c78ed6f8b354b1b9b07151736df1386/users/3e146e6da0294fa78e07d8c1e971af3e/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:56.350563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5553f-b138-4fc1-9cdc-2dc1ccdb9416 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.350927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5553f-b138-4fc1-9cdc-2dc1ccdb9416 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.352492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5553f-b138-4fc1-9cdc-2dc1ccdb9416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.385737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6af5553f-b138-4fc1-9cdc-2dc1ccdb9416 None None] MFA Rules not processed for user `3e146e6da0294fa78e07d8c1e971af3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:56.386986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6af5553f-b138-4fc1-9cdc-2dc1ccdb9416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:56.454569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 989/1993] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:56.466119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:56.469729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:56.530402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737f3b3-cfda-4561-8e3d-43927db9c1de admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:56.533973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81aef5-6620-4427-a4cd-c17f75a7c361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:56.534581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81aef5-6620-4427-a4cd-c17f75a7c361 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:56.534857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81aef5-6620-4427-a4cd-c17f75a7c361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:56.535129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81aef5-6620-4427-a4cd-c17f75a7c361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:56.544328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81aef5-6620-4427-a4cd-c17f75a7c361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:56.545754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81aef5-6620-4427-a4cd-c17f75a7c361 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:56.555039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1005/1994] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:03:57.860438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bea15-e508-44ee-a5a7-288664743c2a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:57.861000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bea15-e508-44ee-a5a7-288664743c2a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:57.861098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bea15-e508-44ee-a5a7-288664743c2a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:57.862907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 990/1995] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:03:57.872832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfdd59-2250-4422-bef3-498fef813b71 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:57.873277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfdd59-2250-4422-bef3-498fef813b71 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:57.873541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfdd59-2250-4422-bef3-498fef813b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:57.892831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2fdfdd59-2250-4422-bef3-498fef813b71 None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:57.894030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2fdfdd59-2250-4422-bef3-498fef813b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:57.937854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bea15-e508-44ee-a5a7-288664743c2a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:57.939729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-417bea15-e508-44ee-a5a7-288664743c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:57.969044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:57.994634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1006/1996] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:03:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:03:57.997027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:57.997809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:57.997809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.028236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] MFA Rules not processed for user `3e146e6da0294fa78e07d8c1e971af3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:58.029688 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:58.030456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bea15-e508-44ee-a5a7-288664743c2a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:58.036245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:58.037186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.037649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.038156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] PATH_INFO: `/v3/users/20ef078fe46d418c95f35e1f0bdd2f2e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.038491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1007/1997] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:58.040859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ef078fe46d418c95f35e1f0bdd2f2e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:58.054020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:58.192108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:58.194212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:58.227123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0c748-7d4e-4b0e-bc68-4be0bc058d96 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:58.229962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ef078fe46d418c95f35e1f0bdd2f2e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.239708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:58.240340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.240620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.240895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.244897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ef078fe46d418c95f35e1f0bdd2f2e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.245479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ef078fe46d418c95f35e1f0bdd2f2e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.260271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:58.264169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ef078fe46d418c95f35e1f0bdd2f2e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.270027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:58.282835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d4ba1445-657e-4ec9-a551-9af50e074c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ef078fe46d418c95f35e1f0bdd2f2e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.284226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1008/1998] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:03:58.291540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 991/1999] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:03:57 2022] DELETE /identity/v3/users/20ef078fe46d418c95f35e1f0bdd2f2e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:03:58.504005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:58.520198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d196f3-96bd-484a-9327-fcdb6113fc79 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:58.574233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:58.599458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:58.599458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.599458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.599458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] PATH_INFO: `/v3/projects/a1b8f338b40e4d7e9a6249084bcdaa73` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.602640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b8f338b40e4d7e9a6249084bcdaa73)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:58.606190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:58.662148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b8f338b40e4d7e9a6249084bcdaa73', 'request_id': 'req-4957c298-54a4-4260-ad47-acca7b39711d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.669941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b8f338b40e4d7e9a6249084bcdaa73', 'request_id': 'req-4957c298-54a4-4260-ad47-acca7b39711d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.676995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b8f338b40e4d7e9a6249084bcdaa73', 'request_id': 'req-4957c298-54a4-4260-ad47-acca7b39711d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.688487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] The token cache is being invalidate because project a1b8f338b40e4d7e9a6249084bcdaa73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:03:58.688775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.689435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4957c298-54a4-4260-ad47-acca7b39711d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:03:58.690922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1009/2000] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:03:58 2022] DELETE /identity/v3/projects/a1b8f338b40e4d7e9a6249084bcdaa73 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:58.697788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.698076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.698343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.720008 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:58.721443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:58.791913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 992/2001] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:58.799509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.799889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.800264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.821393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:58.822967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:58.831484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1010/2002] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:58.835910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:58.863162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea9f28-743d-4f12-90d9-7bf065157575 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:58.868646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:58.869489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.869610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.869805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.870511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:03:58.870757 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:58.872408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:58.885762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 993/2003] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:03:58.890521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:58.900918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99bf7a-ea26-4129-a013-be2ca264058f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:58.907080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:58.907722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:58.908036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:58.908337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:58.909096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:58.910981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:58.946535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1011/2004] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:03:58 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:58.947410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:58.962975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693694c-a935-4bad-970c-620882feb12b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:58.988353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:03:59.037758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:59.038409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:59.038695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:59.039542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:59.040232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:59.042199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:59.079262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 994/2005] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:03:58 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:03:59.080666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:59.092744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b9a1c-61f0-4457-97fc-842069a70e09 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:59.098196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:59.098808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:59.099086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:59.099357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:59.100035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:59.102356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:59.113337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1012/2006] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:59.117748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:59.130473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1dc5-b60f-4183-812a-f2e92f82e515 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:59.136285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850da43-684c-42d6-b817-31bbf157913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:59.136901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850da43-684c-42d6-b817-31bbf157913b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:59.137185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850da43-684c-42d6-b817-31bbf157913b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:59.137460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850da43-684c-42d6-b817-31bbf157913b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:59.138145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850da43-684c-42d6-b817-31bbf157913b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:59.139908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850da43-684c-42d6-b817-31bbf157913b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:59.156693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 995/2007] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:03:59.161397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:59.174572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b3f46-acc6-4344-8600-d005fe0349a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:59.181064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:59.181757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:59.182024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:59.182294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] PATH_INFO: `/v3/projects/435ce18280514d49beb55d8dd51d5876/users/d44100ba230f478a9d3b1fc58431ed58/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:59.201979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] RBAC: Authorizing `identity:create_grant(project_id=435ce18280514d49beb55d8dd51d5876, user_id=d44100ba230f478a9d3b1fc58431ed58, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:59.204829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:59.219471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1013/2008] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:03:59 2022] PUT /identity/v3/projects/435ce18280514d49beb55d8dd51d5876/users/d44100ba230f478a9d3b1fc58431ed58/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:03:59.229742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:59.230030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:59.230334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:59.263394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] MFA Rules not processed for user `d44100ba230f478a9d3b1fc58431ed58`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:03:59.264686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:59.337941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 996/2009] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:03:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:03:59.349801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:03:59.352117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:03:59.436094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d40cd1-aa7f-4516-b54c-ee39d56c608d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:03:59.442169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8eaee9-e5bd-4cac-b62e-c0bcaaac1388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:03:59.442853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8eaee9-e5bd-4cac-b62e-c0bcaaac1388 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:03:59.443533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8eaee9-e5bd-4cac-b62e-c0bcaaac1388 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:03:59.443533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8eaee9-e5bd-4cac-b62e-c0bcaaac1388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:03:59.453413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8eaee9-e5bd-4cac-b62e-c0bcaaac1388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:03:59.455013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8eaee9-e5bd-4cac-b62e-c0bcaaac1388 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:03:59.466265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1014/2010] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:03:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:04:05.474675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:05.561369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b016a-3865-4d0f-8494-b31a17596307 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:05.566397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:05.567171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:05.567549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:05.567926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] PATH_INFO: `/v3/users/d44100ba230f478a9d3b1fc58431ed58` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:05.569215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44100ba230f478a9d3b1fc58431ed58)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:05.571341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:05.615902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:05.626758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44100ba230f478a9d3b1fc58431ed58'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:05.637561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44100ba230f478a9d3b1fc58431ed58'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:05.638161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44100ba230f478a9d3b1fc58431ed58'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:05.646534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44100ba230f478a9d3b1fc58431ed58'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:05.663746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44100ba230f478a9d3b1fc58431ed58'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:05.671651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 997/2011] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:05 2022] DELETE /identity/v3/users/d44100ba230f478a9d3b1fc58431ed58 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:05.676761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:05.678066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:05.678540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:05.694190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:05.701912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:05.703738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:05.785633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1015/2012] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:05.808931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:05.818493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:05.907083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a9203-41d6-42af-857f-a733d16517f6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:05.910769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c279-97cd-4d61-a266-7bdff0553a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:05.911389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c279-97cd-4d61-a266-7bdff0553a75 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:05.911710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c279-97cd-4d61-a266-7bdff0553a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:05.911996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c279-97cd-4d61-a266-7bdff0553a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:05.921943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c279-97cd-4d61-a266-7bdff0553a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:05.923314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c279-97cd-4d61-a266-7bdff0553a75 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:05.934107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 998/2013] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:06.205059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.206535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:06.223787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c27432-ed53-4210-9505-2c88c6af5c2c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.229066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.230012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.230213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.230493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] PATH_INFO: `/v3/projects/435ce18280514d49beb55d8dd51d5876` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.231641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435ce18280514d49beb55d8dd51d5876)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.233566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.280636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435ce18280514d49beb55d8dd51d5876', 'request_id': 'req-4025782d-3518-4fdd-85eb-e84de736b421'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:06.287556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435ce18280514d49beb55d8dd51d5876', 'request_id': 'req-4025782d-3518-4fdd-85eb-e84de736b421'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:06.293676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435ce18280514d49beb55d8dd51d5876', 'request_id': 'req-4025782d-3518-4fdd-85eb-e84de736b421'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:06.303484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] The token cache is being invalidate because project 435ce18280514d49beb55d8dd51d5876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:06.303796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:06.304939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4025782d-3518-4fdd-85eb-e84de736b421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:06.306398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1016/2014] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:06 2022] DELETE /identity/v3/projects/435ce18280514d49beb55d8dd51d5876 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:06.311203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.311491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.311805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.331192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:06.332438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:06.393212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 999/2015] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:06.402181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.402467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.402737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.426322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:06.427508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:06.435805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1017/2016] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:06.439985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.450360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b54bcb-39cb-4f43-bd9c-4ec63f18b079 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.454814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.455409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.455719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.455991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.456633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:06.456936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.458518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.470058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1000/2017] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:06.474197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.484108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b5c61-56fe-4cab-8571-3ade6f217c3e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.488752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.489374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.489654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.489924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.490578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.493122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.519276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1018/2018] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/projects => generated 357 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:06.523368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.533512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15541428-fa32-4ca1-bb2d-afb655e61ade admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.575840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:06.594660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.595277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.595561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.595873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.596531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.598457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.622870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1001/2019] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:06.626963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.637799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c429aa-6bd5-4add-ad96-febfd8cb5383 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.642468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.643218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.643494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.643795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.644444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.646007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.659489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1019/2020] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:06.660812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.672149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcab15-60b9-43d3-a219-95f8ad28f170 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.677188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1712fc-c1d9-45f4-86b3-3cabd27cb4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.677798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1712fc-c1d9-45f4-86b3-3cabd27cb4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.678078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1712fc-c1d9-45f4-86b3-3cabd27cb4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.678351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1712fc-c1d9-45f4-86b3-3cabd27cb4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.679023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1712fc-c1d9-45f4-86b3-3cabd27cb4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.681592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1712fc-c1d9-45f4-86b3-3cabd27cb4ce admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.692176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1002/2021] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:06.696629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.706978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64912460-84fb-4894-8116-12bb95e80f57 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.711995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.712674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.712927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.713208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] PATH_INFO: `/v3/projects/1525deb5737e48719c058c1ed105dc79/users/3aa475b31e1a48ccaadc3c20c42f6cd2/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.728600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1525deb5737e48719c058c1ed105dc79, user_id=3aa475b31e1a48ccaadc3c20c42f6cd2, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.731251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.745261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1020/2022] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:06 2022] PUT /identity/v3/projects/1525deb5737e48719c058c1ed105dc79/users/3aa475b31e1a48ccaadc3c20c42f6cd2/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:06.755860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.756159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.756439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.785392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] MFA Rules not processed for user `3aa475b31e1a48ccaadc3c20c42f6cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:06.786657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:06.853742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1003/2023] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:06.859767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.892093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:06.930761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7c3d1-85d7-4bbe-a5b6-5f1674ccc0cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:06.935572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:06.936277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:06.936627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:06.936906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:06.937535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:06.939431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:06.969726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1021/2024] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/projects => generated 355 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:06.971077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:06.982737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2da0ed-e6b7-4a91-9d14-9267d6617d48 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.041731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.042336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.042614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.042884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.043498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.045438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.066883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1004/2025] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:06 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:07.070964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:07.081184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f712c6-4d3c-4f4c-bb68-bf032504fbef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.086412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.087106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.087388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.087703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.088350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.089941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.100224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1022/2026] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:07.105133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:07.114909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e484d8-1afc-4318-9972-321cd11cb5d4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.119571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.120247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.120528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.120806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] PATH_INFO: `/v3/projects/31134935efcc42a98d9f56987c751126/users/95122895076f4843ad1d32a7a0f0d519/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.135330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31134935efcc42a98d9f56987c751126, user_id=95122895076f4843ad1d32a7a0f0d519, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.138022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.151322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1005/2027] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:07 2022] PUT /identity/v3/projects/31134935efcc42a98d9f56987c751126/users/95122895076f4843ad1d32a7a0f0d519/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:07.155293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:07.165692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedbf59-38d3-49b6-9a33-685675d0f24f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.221056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3e90f-5eac-4e96-a114-c7578e83dea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.221585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3e90f-5eac-4e96-a114-c7578e83dea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.221861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3e90f-5eac-4e96-a114-c7578e83dea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.222132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3e90f-5eac-4e96-a114-c7578e83dea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.222765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3e90f-5eac-4e96-a114-c7578e83dea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.224377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3e90f-5eac-4e96-a114-c7578e83dea6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.234601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1023/2028] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:07.238963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:07.249532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a554-b1c0-4507-9ee2-0c8c53c2f722 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.254522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.255168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.255445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.255752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] PATH_INFO: `/v3/projects/31134935efcc42a98d9f56987c751126/users/95122895076f4843ad1d32a7a0f0d519/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.258170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=31134935efcc42a98d9f56987c751126, user_id=95122895076f4843ad1d32a7a0f0d519, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.260805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.270711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1006/2029] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:07 2022] PUT /identity/v3/projects/31134935efcc42a98d9f56987c751126/users/95122895076f4843ad1d32a7a0f0d519/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:07.281285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.281574 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.281847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.309334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] MFA Rules not processed for user `95122895076f4843ad1d32a7a0f0d519`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:07.310504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:07.375749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1024/2030] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:07.387779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:07.389465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:07.394730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:07.450866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a8177-8c31-4024-83a2-daeb443c9a8b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.456070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a681f-125d-43b1-bd66-a1deed1ed16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.456772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a681f-125d-43b1-bd66-a1deed1ed16b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.457049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a681f-125d-43b1-bd66-a1deed1ed16b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.457319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a681f-125d-43b1-bd66-a1deed1ed16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.467746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a681f-125d-43b1-bd66-a1deed1ed16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.469344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a681f-125d-43b1-bd66-a1deed1ed16b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.479655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1007/2031] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:07.552313 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:07.561635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d3183-9e03-41f3-bb74-d67dd1014743 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:07.565221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:07.565808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:07.566082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:07.566350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:07.579289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:07.625854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:07.627273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:07.637106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1025/2032] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:08.845868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2074c1-11b3-499c-bba7-fadec634f139 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:08.846353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2074c1-11b3-499c-bba7-fadec634f139 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:08.846626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2074c1-11b3-499c-bba7-fadec634f139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:08.873683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-de2074c1-11b3-499c-bba7-fadec634f139 None None] MFA Rules not processed for user `2a6b55d7cb1840a5ad93369589a36596`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:08.875042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-de2074c1-11b3-499c-bba7-fadec634f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:08.943010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1008/2033] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:13.335160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:13.336602 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:13.401586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ec8bf-4032-4d96-adbf-6ab10cde4566 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:13.406520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:13.407142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:13.407424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:13.407728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] PATH_INFO: `/v3/users/16b01171dff342bfb599ded091a78b86` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:13.408873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b01171dff342bfb599ded091a78b86)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:13.410812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:13.461645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b01171dff342bfb599ded091a78b86'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:13.470437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b01171dff342bfb599ded091a78b86'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:13.478557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b01171dff342bfb599ded091a78b86'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:13.479217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b01171dff342bfb599ded091a78b86'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:13.492336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b01171dff342bfb599ded091a78b86'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:13.498301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1026/2034] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:13 2022] DELETE /identity/v3/users/16b01171dff342bfb599ded091a78b86 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:13.502775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:13.503259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:13.503479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:13.522855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:13.524209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:13.586139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1009/2035] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:13.596534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:13.599770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:13.662745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:13.687341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004cd1-2c0d-4280-a6b1-d47aa5c50102 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:13.691184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:13.691918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:13.692280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:13.692637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:13.702762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:13.704278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:13.717264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1027/2036] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:14.364144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:14.367168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:14.382987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57297-e150-4372-96a9-5c56e6e288ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:14.390654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:14.391537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:14.391698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:14.391898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] PATH_INFO: `/v3/projects/646ce05103f943c4943133590ccf783b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:14.393087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646ce05103f943c4943133590ccf783b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:14.395017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:14.468231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:14.471098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646ce05103f943c4943133590ccf783b', 'request_id': 'req-5b782e7b-1111-4b93-bea7-8964d9f2b958'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.480801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646ce05103f943c4943133590ccf783b', 'request_id': 'req-5b782e7b-1111-4b93-bea7-8964d9f2b958'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.488039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646ce05103f943c4943133590ccf783b', 'request_id': 'req-5b782e7b-1111-4b93-bea7-8964d9f2b958'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.503378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] The token cache is being invalidate because project 646ce05103f943c4943133590ccf783b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:14.504021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.504801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.506633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1010/2037] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:14 2022] DELETE /identity/v3/projects/646ce05103f943c4943133590ccf783b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:14.510445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:14.521132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3ff30-bfa9-4724-81be-b92645dd63fc service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:14.591335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:14.592007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:14.592284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:14.592556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] PATH_INFO: `/v3/users/dd4c69e3e0714a67908bb4de65b42d89` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:14.593681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4c69e3e0714a67908bb4de65b42d89)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:14.595669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:14.659773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4c69e3e0714a67908bb4de65b42d89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.669365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4c69e3e0714a67908bb4de65b42d89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.678058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4c69e3e0714a67908bb4de65b42d89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.678748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4c69e3e0714a67908bb4de65b42d89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.696900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0538908-f96d-4162-97f5-aa558c4b1eb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4c69e3e0714a67908bb4de65b42d89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:14.704605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1028/2038] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:14 2022] DELETE /identity/v3/users/dd4c69e3e0714a67908bb4de65b42d89 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:14.999027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.000601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:15.015878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782e7b-1111-4b93-bea7-8964d9f2b958 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.073186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.073825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.074110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.074384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] PATH_INFO: `/v3/projects/595ef7ec4f1742c195d4f9bdfe615a0d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.075516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=595ef7ec4f1742c195d4f9bdfe615a0d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.077501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.134856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '595ef7ec4f1742c195d4f9bdfe615a0d', 'request_id': 'req-bd8e112a-fd53-4908-9896-902c9d165239'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:15.140520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '595ef7ec4f1742c195d4f9bdfe615a0d', 'request_id': 'req-bd8e112a-fd53-4908-9896-902c9d165239'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:15.147420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '595ef7ec4f1742c195d4f9bdfe615a0d', 'request_id': 'req-bd8e112a-fd53-4908-9896-902c9d165239'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:15.164576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] The token cache is being invalidate because project 595ef7ec4f1742c195d4f9bdfe615a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:15.164867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:15.165508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd8e112a-fd53-4908-9896-902c9d165239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:15.167089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1011/2039] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:14 2022] DELETE /identity/v3/projects/595ef7ec4f1742c195d4f9bdfe615a0d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:15.171868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.172488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.172772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.198832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:15.200107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:15.251859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:15.264128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1029/2040] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:15.273133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.273441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.273717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.293704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:15.294946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:15.303025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1012/2041] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:15.308003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.317624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4859d7-a85c-4541-a588-d856198e6a10 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.323458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.324109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.324388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.324652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.325305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:15.325621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.327217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.345260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1030/2042] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:15.349692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.358924 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daad056-ab71-4d79-9e68-06d900810765 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.363611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.364227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.364506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.364778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.365406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.367918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.396967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1013/2043] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:15.401596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.411494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a2b75-2ed2-4548-8938-f472ed30c4a2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.468108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.468724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.469008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.469281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.469918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.471856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.494982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1031/2044] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:15.499505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.512736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614d7c-e10c-436d-bddc-8d790b73a6f6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.518601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.519221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.519503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.519804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.520449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.522164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.533203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1014/2045] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:15.537046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.547797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd2094-3d12-49cf-8a7f-1cfe51f2b8c6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.552659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d7fd0-e221-4a6f-8ee6-1f476366bddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.553354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7fd0-e221-4a6f-8ee6-1f476366bddb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.553702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7fd0-e221-4a6f-8ee6-1f476366bddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.554677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7fd0-e221-4a6f-8ee6-1f476366bddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.555336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d7fd0-e221-4a6f-8ee6-1f476366bddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.556983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d7fd0-e221-4a6f-8ee6-1f476366bddb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.568486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1032/2046] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:15.572945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.583170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f002a0-a980-4fe5-b2d4-f36c7f8786fc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.587992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.588650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.588932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.589208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] PATH_INFO: `/v3/projects/a1382691cd444b8db789045a579b89d2/users/c20a424044fa4dd6bb5fdda400d89b98/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.605476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1382691cd444b8db789045a579b89d2, user_id=c20a424044fa4dd6bb5fdda400d89b98, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.608956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.618771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:15.624678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1015/2047] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:15 2022] PUT /identity/v3/projects/a1382691cd444b8db789045a579b89d2/users/c20a424044fa4dd6bb5fdda400d89b98/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:15.635243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.635606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.635886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.666138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] MFA Rules not processed for user `c20a424044fa4dd6bb5fdda400d89b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:15.667364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:15.736498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1033/2048] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:15.740940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.810551 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54366952-5516-498f-b79b-1387ea843e4b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.815233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.815876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.816160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.816505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.817145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.819042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.849973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1016/2049] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:15.854710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.865857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45f59-ae05-4489-8e61-633f3be91b5a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.915528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:15.927407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.928064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.928348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.928620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.929826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.931845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.957693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1034/2050] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:15 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:15.962226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:15.972900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d315f-43f7-4f48-b25f-8275c7e19008 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:15.978449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:15.979138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:15.979501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:15.979749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:15.980515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:15.982325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:15.994081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1017/2051] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:15.998101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:16.009700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f56384-76c7-40da-83b9-56e9209f4ccf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:16.015084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:16.015892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:16.016225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:16.016504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] PATH_INFO: `/v3/projects/c1f49c726ffb4afc83cfd15ae1f0a9cc/users/3b77b36aa67c4f8799e6eb924881114c/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:16.032822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f49c726ffb4afc83cfd15ae1f0a9cc, user_id=3b77b36aa67c4f8799e6eb924881114c, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:16.035588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:16.049154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1035/2052] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:15 2022] PUT /identity/v3/projects/c1f49c726ffb4afc83cfd15ae1f0a9cc/users/3b77b36aa67c4f8799e6eb924881114c/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:16.056091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:16.068218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a04c7-6646-4ace-bccd-aaef019b2034 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:16.157305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3be009-7546-44f1-8d85-81fce4cc387e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:16.158129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be009-7546-44f1-8d85-81fce4cc387e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:16.158510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be009-7546-44f1-8d85-81fce4cc387e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:16.159016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be009-7546-44f1-8d85-81fce4cc387e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:16.159901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3be009-7546-44f1-8d85-81fce4cc387e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:16.161983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3be009-7546-44f1-8d85-81fce4cc387e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:16.175966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1018/2053] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:16.180863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:16.199840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49565de1-9d14-49ca-a374-c19f91879414 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:16.205780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:16.206461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:16.206740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:16.207013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] PATH_INFO: `/v3/projects/c1f49c726ffb4afc83cfd15ae1f0a9cc/users/3b77b36aa67c4f8799e6eb924881114c/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:16.209014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f49c726ffb4afc83cfd15ae1f0a9cc, user_id=3b77b36aa67c4f8799e6eb924881114c, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:16.211668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:16.223364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1036/2054] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:16 2022] PUT /identity/v3/projects/c1f49c726ffb4afc83cfd15ae1f0a9cc/users/3b77b36aa67c4f8799e6eb924881114c/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:16.238681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:16.238978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:16.239256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:16.269708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] MFA Rules not processed for user `3b77b36aa67c4f8799e6eb924881114c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:16.270927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:16.330329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:16.340734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1019/2055] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:16.353150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:16.354874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:16.425678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4771c-5dca-491f-a5d0-e8c1454387ac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:16.430543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:16.431414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:16.431860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:16.432499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:16.443351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:16.446122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:16.458494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1037/2056] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:16.517740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:16.519199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:16.531978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df956f-e385-4365-b5d3-0264a977c265 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:16.537707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5879126-5cc8-4c27-9a72-ca9fa283bef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:16.538964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879126-5cc8-4c27-9a72-ca9fa283bef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:16.539744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879126-5cc8-4c27-9a72-ca9fa283bef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:16.539744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879126-5cc8-4c27-9a72-ca9fa283bef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:16.619054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5879126-5cc8-4c27-9a72-ca9fa283bef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:16.620555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5879126-5cc8-4c27-9a72-ca9fa283bef7 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:16.630799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1020/2057] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:20.066465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:20.181040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff3afc-073f-49df-8b0b-4f1281df4d37 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:20.185804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:20.186459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:20.186731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:20.187008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] PATH_INFO: `/v3/users/c20a424044fa4dd6bb5fdda400d89b98` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:20.188202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20a424044fa4dd6bb5fdda400d89b98)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:20.190775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:20.275395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20a424044fa4dd6bb5fdda400d89b98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.279266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:20.288869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20a424044fa4dd6bb5fdda400d89b98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.308679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20a424044fa4dd6bb5fdda400d89b98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.309719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20a424044fa4dd6bb5fdda400d89b98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.335842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20a424044fa4dd6bb5fdda400d89b98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.359750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1038/2058] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:20 2022] DELETE /identity/v3/users/c20a424044fa4dd6bb5fdda400d89b98 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:20.359889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:20.360308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:20.360396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:20.391874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:20.393137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:20.475762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1021/2059] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:20.490265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:20.491870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:20.565173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a02163-ace4-4ebe-ac93-7bda7ffe55fd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:20.569302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:20.569983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:20.570326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:20.571279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:20.582406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:20.583868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:20.594535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1039/2060] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:20.877165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:20.878669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:20.893773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c3e8a-ae33-4a4e-ac81-fbbbfd11624b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:20.900020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:20.900748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:20.901093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:20.901457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] PATH_INFO: `/v3/projects/a1382691cd444b8db789045a579b89d2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:20.902650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1382691cd444b8db789045a579b89d2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:20.905413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:20.952883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1382691cd444b8db789045a579b89d2', 'request_id': 'req-8acf1c51-9954-48ac-8962-d74e75c321e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.958799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1382691cd444b8db789045a579b89d2', 'request_id': 'req-8acf1c51-9954-48ac-8962-d74e75c321e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.964943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1382691cd444b8db789045a579b89d2', 'request_id': 'req-8acf1c51-9954-48ac-8962-d74e75c321e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.977113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] The token cache is being invalidate because project a1382691cd444b8db789045a579b89d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:20.977638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.978355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:20.979862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1022/2061] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:20 2022] DELETE /identity/v3/projects/a1382691cd444b8db789045a579b89d2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:20.983890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:20.994253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f084310-73be-4014-81b4-86a9bdc6f951 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:21.050365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:21.050922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.051200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.051473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] PATH_INFO: `/v3/users/3b77b36aa67c4f8799e6eb924881114c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.052684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b77b36aa67c4f8799e6eb924881114c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:21.054632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:21.100919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b77b36aa67c4f8799e6eb924881114c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.111495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b77b36aa67c4f8799e6eb924881114c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.118664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b77b36aa67c4f8799e6eb924881114c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.119303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b77b36aa67c4f8799e6eb924881114c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.126183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:21.132383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60785f1e-03a2-4842-af09-44dd952179a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b77b36aa67c4f8799e6eb924881114c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.139207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1040/2062] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:20 2022] DELETE /identity/v3/users/3b77b36aa67c4f8799e6eb924881114c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:21.460912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:21.463991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:21.482749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf1c51-9954-48ac-8962-d74e75c321e3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:21.497232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:21.556225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:21.557038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.557665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.558227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] PATH_INFO: `/v3/projects/c1f49c726ffb4afc83cfd15ae1f0a9cc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.573894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f49c726ffb4afc83cfd15ae1f0a9cc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:21.575871 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:21.636232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f49c726ffb4afc83cfd15ae1f0a9cc', 'request_id': 'req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.650391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f49c726ffb4afc83cfd15ae1f0a9cc', 'request_id': 'req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.652363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f49c726ffb4afc83cfd15ae1f0a9cc', 'request_id': 'req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.666044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] The token cache is being invalidate because project c1f49c726ffb4afc83cfd15ae1f0a9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:21.666366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.667003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3b86fc99-9e51-445e-ade6-f4ea4fc564f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:21.668533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1023/2063] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:21 2022] DELETE /identity/v3/projects/c1f49c726ffb4afc83cfd15ae1f0a9cc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:21.673530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.673831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.674094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.693696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:21.695072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:21.761061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1041/2064] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:21.768771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.769057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.769330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.790389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:21.791695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:21.800192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1024/2065] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:21.804424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:21.814655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a89f-0b6f-477d-b4d9-757cccd4e589 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:21.819932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:21.821040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.821397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.821596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.822264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:21.822584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:21.824355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:21.836085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1042/2066] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:21.841577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:21.851229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ba853-b4b6-4d65-9420-b99692c8ff40 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:21.856144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:21.856790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.857065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.857331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.857944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:21.859836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:21.888770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1025/2067] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:21 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:21.893016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:21.903736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7e7f-6893-406d-ba6d-815a8a7ed810 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:21.969694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:21.970352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:21.970583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:21.970855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:21.971488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:21.973421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:21.998514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1043/2068] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:21 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:22.000305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.013429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6692d-3d1f-43ea-ab20-f1fbb5498e47 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.018368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.019404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.019404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.019641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.020179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.022014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.033729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1026/2069] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:22.042010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.053264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f59e0-0e0d-418e-824c-a8017399e195 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.060232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c172a-c5d2-4130-a53d-9842ddf2965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.060840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c172a-c5d2-4130-a53d-9842ddf2965a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.061144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c172a-c5d2-4130-a53d-9842ddf2965a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.061415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c172a-c5d2-4130-a53d-9842ddf2965a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.062149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c172a-c5d2-4130-a53d-9842ddf2965a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.063885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c172a-c5d2-4130-a53d-9842ddf2965a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.076923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1044/2070] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:22.081039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.092486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55a425-45dd-40e3-9285-4d3447a0599e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.097845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.098485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.098845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.099175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] PATH_INFO: `/v3/projects/e2da6c60426c43f485445c421dce1249/users/0098866354f04715b5829a927660ab7a/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.115091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2da6c60426c43f485445c421dce1249, user_id=0098866354f04715b5829a927660ab7a, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.118798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.132057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1027/2071] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:22 2022] PUT /identity/v3/projects/e2da6c60426c43f485445c421dce1249/users/0098866354f04715b5829a927660ab7a/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:22.143189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.143689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.143784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.170875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] MFA Rules not processed for user `0098866354f04715b5829a927660ab7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:22.172133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:22.215896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:22.237826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1045/2072] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:22 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:22.242861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.288264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:22.306748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094a0f-a66f-4e84-b9a8-56d5d1e616ae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.311421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.312062 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.312343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.312627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.313250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.315143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.342903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1028/2073] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:22 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:22.346912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.358016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3883a-bbd0-49c7-a6fb-8db8ab6f641d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.413233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.413872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.414666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.414946 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.415564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.417518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.438786 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1046/2074] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:22 2022] POST /identity/v3/users => generated 307 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:22.442779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.453323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b636b-bf5d-449f-9195-6cbf98c233d3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.457947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.458539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.458817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.459107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.459770 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.461453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.471813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1029/2075] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:22.475749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.486109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a5f0-3751-4b91-893d-001a5beca8fa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.490656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.491297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.491582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.491886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] PATH_INFO: `/v3/projects/4e7a559007464521b8527e0b73004e9e/users/16e8f2d9c877406aaf3508ac9a887c16/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.507099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e7a559007464521b8527e0b73004e9e, user_id=16e8f2d9c877406aaf3508ac9a887c16, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.510272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.526814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1047/2076] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:22 2022] PUT /identity/v3/projects/4e7a559007464521b8527e0b73004e9e/users/16e8f2d9c877406aaf3508ac9a887c16/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:22.530816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.541280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2156-491a-4474-883d-a278b28a6f9c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.598548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986f9b-2ecd-48ea-b023-8740b688110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.599156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986f9b-2ecd-48ea-b023-8740b688110e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.599433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986f9b-2ecd-48ea-b023-8740b688110e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.599747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986f9b-2ecd-48ea-b023-8740b688110e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.600492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c986f9b-2ecd-48ea-b023-8740b688110e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.603200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c986f9b-2ecd-48ea-b023-8740b688110e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.615727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1030/2077] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:22.620067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.630750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6570-8e41-471b-98d6-f6ca1c7112ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.635466 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.636234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.636539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.636805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] PATH_INFO: `/v3/projects/4e7a559007464521b8527e0b73004e9e/users/16e8f2d9c877406aaf3508ac9a887c16/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.638753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e7a559007464521b8527e0b73004e9e, user_id=16e8f2d9c877406aaf3508ac9a887c16, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.641429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.652938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1048/2078] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:22 2022] PUT /identity/v3/projects/4e7a559007464521b8527e0b73004e9e/users/16e8f2d9c877406aaf3508ac9a887c16/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:22.665633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.665633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.665633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.696743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] MFA Rules not processed for user `16e8f2d9c877406aaf3508ac9a887c16`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:22.697959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:22.784861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1031/2079] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:22 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:22.793718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:22.795515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:22.877418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cd4cf-8ed2-4a8f-97fa-d9984ea99c14 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:22.881960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:22.882644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:22.883573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:22.883950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:22.898342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:22.900055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:22.930863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1049/2080] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:22 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:23.001679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:23.003114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:23.031486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:23.033179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:23.048302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa2654-43d4-4920-93da-2a983fda43af service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:23.052341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36b05e-ad71-4a60-9ce5-f18e10282263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:23.052948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b05e-ad71-4a60-9ce5-f18e10282263 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:23.053223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b05e-ad71-4a60-9ce5-f18e10282263 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:23.053492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b05e-ad71-4a60-9ce5-f18e10282263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:23.056282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:23.085313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a45605-f88f-4f80-af2d-d918dfc8912a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:23.091626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:23.092500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:23.092846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:23.093430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] PATH_INFO: `/v3/users/3aa475b31e1a48ccaadc3c20c42f6cd2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:23.094714 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa475b31e1a48ccaadc3c20c42f6cd2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:23.096959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:23.098317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:23.122245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36b05e-ad71-4a60-9ce5-f18e10282263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:23.124029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36b05e-ad71-4a60-9ce5-f18e10282263 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:23.138384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1050/2081] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:23 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:04:23.160983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa475b31e1a48ccaadc3c20c42f6cd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.168511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa475b31e1a48ccaadc3c20c42f6cd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.169316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa475b31e1a48ccaadc3c20c42f6cd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.182721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa475b31e1a48ccaadc3c20c42f6cd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.207940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa475b31e1a48ccaadc3c20c42f6cd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.217317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1032/2082] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:23 2022] DELETE /identity/v3/users/3aa475b31e1a48ccaadc3c20c42f6cd2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:23.225495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:23.226006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:23.226792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:23.249531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:23.250945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:23.326134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1051/2083] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:23.340943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:23.342532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:23.476355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde2c1-f58f-4fd5-839d-7b4ed5541cdd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:23.481383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:23.482000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:23.482280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:23.482549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:23.496550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:23.499978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:23.544264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1033/2084] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:23.859894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:23.859894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:23.877486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17e61f-21da-4ccc-a394-be164513a708 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:23.882633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:23.883464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:23.883970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:23.884248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] PATH_INFO: `/v3/projects/1525deb5737e48719c058c1ed105dc79` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:23.885388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1525deb5737e48719c058c1ed105dc79)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:23.887381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:23.939259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1525deb5737e48719c058c1ed105dc79', 'request_id': 'req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.949700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1525deb5737e48719c058c1ed105dc79', 'request_id': 'req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.956237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1525deb5737e48719c058c1ed105dc79', 'request_id': 'req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.966937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] The token cache is being invalidate because project 1525deb5737e48719c058c1ed105dc79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:23.967218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.967862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:23.969335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1052/2085] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:23 2022] DELETE /identity/v3/projects/1525deb5737e48719c058c1ed105dc79 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:23.973363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:23.983560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9493ac8-77cd-4cfb-8c9e-182adc9e7f6b service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:24.054220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:24.056065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:24.056348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:24.056658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] PATH_INFO: `/v3/users/95122895076f4843ad1d32a7a0f0d519` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:24.057962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95122895076f4843ad1d32a7a0f0d519)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:24.059989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:24.063072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:24.119061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95122895076f4843ad1d32a7a0f0d519'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.129696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95122895076f4843ad1d32a7a0f0d519'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.130682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95122895076f4843ad1d32a7a0f0d519'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.139889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95122895076f4843ad1d32a7a0f0d519'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.158081 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3b798f9-a16c-41fa-b82c-94ed2c066336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95122895076f4843ad1d32a7a0f0d519'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.167005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1034/2086] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:23 2022] DELETE /identity/v3/users/95122895076f4843ad1d32a7a0f0d519 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:24.471846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:24.473604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:24.473894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:24.474168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:24.477461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:24.503001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6b1a-a972-48b0-8eab-75baef5d6ff9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:24.505080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:24.506560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:24.590554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1035/2087] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:04:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:24.605835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:24.607733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:24.608384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:24.608892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] PATH_INFO: `/v3/projects/31134935efcc42a98d9f56987c751126` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:24.611075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31134935efcc42a98d9f56987c751126)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:24.613138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:24.615899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:24.652373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:24.653991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:24.717137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31134935efcc42a98d9f56987c751126', 'request_id': 'req-75819a3e-abdb-482d-ab10-aa45fc4306d6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.732636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31134935efcc42a98d9f56987c751126', 'request_id': 'req-75819a3e-abdb-482d-ab10-aa45fc4306d6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.736295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97330ec0-9b55-45ec-a6c9-b763db37ceaa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:24.747906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31134935efcc42a98d9f56987c751126', 'request_id': 'req-75819a3e-abdb-482d-ab10-aa45fc4306d6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.782788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:24.795095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] The token cache is being invalidate because project 31134935efcc42a98d9f56987c751126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:24.795435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.796257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75819a3e-abdb-482d-ab10-aa45fc4306d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:24.801624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1053/2088] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:24 2022] DELETE /identity/v3/projects/31134935efcc42a98d9f56987c751126 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:24.804159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:24.804286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:24.804618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:24.866888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:24.868460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:24.868958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:24.869437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:24.872093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:24.884613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:24.886059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:24.941690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:24.943455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f47a53-f639-43c4-94b6-29b3d5f2a11f service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:24.953421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1036/2089] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:04:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:04:24.986336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1054/2090] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:24.995489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:24.996210 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:24.996686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.053063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:25.054574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:25.075580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1037/2091] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:25.080095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.093637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7af8-55cd-4a6e-8c57-de2457b6ef87 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.098897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.099752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.100183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.100715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.101374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:25.101722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.103382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.120256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1055/2092] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:25.122027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.135228 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75dabe-81f9-4df9-8154-a693c074fd00 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.140789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.141917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.142382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.142927 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.143773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.146366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.183915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1038/2093] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:25 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:25.185510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.199777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcf63c-0112-4622-a897-073e71239a56 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.298499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.299245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.299623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.299943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.300587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.302516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.324718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1056/2094] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:25 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:25.329971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.341653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2deea8-13e3-4d8e-b10c-32fd1d1c10d6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.347110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.347768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.348057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.348332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.349180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.350925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.364230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1039/2095] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:25.365490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.376040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c0413-e97a-4275-a2f5-fd54b8889f79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.380977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313b44b-792c-4266-aed6-5b62b70f5725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.381613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313b44b-792c-4266-aed6-5b62b70f5725 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.381897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313b44b-792c-4266-aed6-5b62b70f5725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.382176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313b44b-792c-4266-aed6-5b62b70f5725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.383378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313b44b-792c-4266-aed6-5b62b70f5725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.385022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313b44b-792c-4266-aed6-5b62b70f5725 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.387302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:25.399967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1057/2096] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:25.403693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.413786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8ca07-9b9f-4574-bc4a-560eb249497e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.419475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.420160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.420436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.420710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] PATH_INFO: `/v3/projects/29b8b3474cd94a59a848ecb539898023/users/2834475488344152bd6b9efb3097c639/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.436391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b8b3474cd94a59a848ecb539898023, user_id=2834475488344152bd6b9efb3097c639, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.439629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.453809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1040/2097] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:25 2022] PUT /identity/v3/projects/29b8b3474cd94a59a848ecb539898023/users/2834475488344152bd6b9efb3097c639/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:25.464614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44b89b-2398-4709-8c26-4ea781421cea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.464898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44b89b-2398-4709-8c26-4ea781421cea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.465175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44b89b-2398-4709-8c26-4ea781421cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.494229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e44b89b-2398-4709-8c26-4ea781421cea None None] MFA Rules not processed for user `2834475488344152bd6b9efb3097c639`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:25.495443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e44b89b-2398-4709-8c26-4ea781421cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:25.561410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1058/2098] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:25.565828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.639820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afff9d-54c0-4cff-813f-ab75c35672df admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.644546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.645190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.645485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.645896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.646529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.648652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.655902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.656188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.656488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.660111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:25.676918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:25.678104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:25.678409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1041/2099] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:25 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:25.683069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.739792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1059/2100] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 9 07:04:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:25.746826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bd84a-6115-4053-a39b-218ed737fb54 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.751491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.752132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.752411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.752689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.753334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.753639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.755253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:25.755519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.778642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1042/2101] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:25 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:25.782492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.793805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e843-ed01-4a3f-bb32-7bd06f86a9e6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.798494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.799086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.799363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.799654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.800305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.801995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.813065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1043/2102] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:25.817053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.827280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86d4c5-09d3-4d10-a6ed-8fbf2351f087 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.831326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913b52-4c81-45fd-910c-74a7cfd0b85d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.832141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.832778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.833059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.833332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] PATH_INFO: `/v3/projects/fa65e81a15624b3da74c9bfe483126fc/users/b592083182c9470cac99e30a37481dda/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.842755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.843386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.843713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.843965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.848732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa65e81a15624b3da74c9bfe483126fc, user_id=b592083182c9470cac99e30a37481dda, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.851384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.852708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.854306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.863373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1060/2103] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:04:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 07:04:25.866252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1044/2104] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:25 2022] PUT /identity/v3/projects/fa65e81a15624b3da74c9bfe483126fc/users/b592083182c9470cac99e30a37481dda/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:25.870782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.881814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f9b8-d554-449e-8319-18ffc09f2e24 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.901534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:25.942834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87db72-ce54-4c24-9edd-cc1c785b64a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.943590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87db72-ce54-4c24-9edd-cc1c785b64a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.944001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87db72-ce54-4c24-9edd-cc1c785b64a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.944375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87db72-ce54-4c24-9edd-cc1c785b64a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.945175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87db72-ce54-4c24-9edd-cc1c785b64a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.947367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87db72-ce54-4c24-9edd-cc1c785b64a6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.957293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1061/2105] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:25.961833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:25.972637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14ac62-8c16-455e-a197-83f228c80ba3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:25.978864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:25.979703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:25.980091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:25.980432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] PATH_INFO: `/v3/projects/fa65e81a15624b3da74c9bfe483126fc/users/b592083182c9470cac99e30a37481dda/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:25.982557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa65e81a15624b3da74c9bfe483126fc, user_id=b592083182c9470cac99e30a37481dda, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:25.985222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:25.997053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1045/2106] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:25 2022] PUT /identity/v3/projects/fa65e81a15624b3da74c9bfe483126fc/users/b592083182c9470cac99e30a37481dda/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:26.008450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cb208-3942-4008-a278-6a10ec0caa8b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.008762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cb208-3942-4008-a278-6a10ec0caa8b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.009037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cb208-3942-4008-a278-6a10ec0caa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.038589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a97cb208-3942-4008-a278-6a10ec0caa8b None None] MFA Rules not processed for user `b592083182c9470cac99e30a37481dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:26.040679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a97cb208-3942-4008-a278-6a10ec0caa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:26.116984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1062/2107] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:26.130674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:26.132442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:26.137126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.137479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.138166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.166761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:26.168060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:26.194870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21df6b-fc4a-4f68-8bcf-3d9d6c855a80 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:26.198846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:26.199465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.199819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.200130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.215306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:26.216811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:26.227754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1046/2108] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:26.238181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1063/2109] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:26.251096 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:26.319490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:26.347987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:26.396798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c5113-78b4-454a-8437-1a2e53b40322 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:26.404460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:26.406564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247ba8e-2604-4b33-ab46-dcb213f6122a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:26.407390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247ba8e-2604-4b33-ab46-dcb213f6122a service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.408032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247ba8e-2604-4b33-ab46-dcb213f6122a service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.408479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247ba8e-2604-4b33-ab46-dcb213f6122a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.425147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c55f2-aadf-4b1e-a29e-edeef0d75348 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:26.432643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:26.433296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.433577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.433848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] PATH_INFO: `/v3/users/2834475488344152bd6b9efb3097c639` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.435188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2834475488344152bd6b9efb3097c639)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:26.435986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247ba8e-2604-4b33-ab46-dcb213f6122a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:26.437541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:26.437677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247ba8e-2604-4b33-ab46-dcb213f6122a service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:26.458238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1047/2110] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:04:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 07:04:26.509076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2834475488344152bd6b9efb3097c639'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:26.517810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2834475488344152bd6b9efb3097c639'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:26.533322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2834475488344152bd6b9efb3097c639'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:26.534201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2834475488344152bd6b9efb3097c639'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:26.551248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2834475488344152bd6b9efb3097c639'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:26.558180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1064/2111] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:26 2022] DELETE /identity/v3/users/2834475488344152bd6b9efb3097c639 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:26.573521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ab114-0913-4f8f-bfb3-1761f0e72bf5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.574092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ab114-0913-4f8f-bfb3-1761f0e72bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.574092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ab114-0913-4f8f-bfb3-1761f0e72bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.597631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-118ab114-0913-4f8f-bfb3-1761f0e72bf5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:26.598849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-118ab114-0913-4f8f-bfb3-1761f0e72bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:26.697264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1048/2112] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:26.708357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:26.710141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:26.781146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7ec70-9d0d-4b3f-8a81-acab625fc74e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:26.785745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:26.786350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.786626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.786915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.802454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:26.804422 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:26.817161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1065/2113] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:26.863079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:26.863372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:26.863676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:26.885159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:26.886542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:26.956764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1049/2114] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 9 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:26.972306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:26.973171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:27.062535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3bc7d-49bb-49fd-915b-279b1d8e5888 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:27.072635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:27.073304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:27.073593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:27.073983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:27.088177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:27.089779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:27.095539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:27.100009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1066/2115] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:04:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:04:27.138031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:27.139922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:27.156514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09644c4b-7d19-4245-aa01-dd1b3b7d1de1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:27.161643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:27.162441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:27.162727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:27.163002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] PATH_INFO: `/v3/projects/29b8b3474cd94a59a848ecb539898023` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:27.164970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b8b3474cd94a59a848ecb539898023)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:27.166936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:27.218592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b8b3474cd94a59a848ecb539898023', 'request_id': 'req-16dee14a-eb61-41b6-aa32-8d57d6660369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.225205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b8b3474cd94a59a848ecb539898023', 'request_id': 'req-16dee14a-eb61-41b6-aa32-8d57d6660369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.233003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b8b3474cd94a59a848ecb539898023', 'request_id': 'req-16dee14a-eb61-41b6-aa32-8d57d6660369'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.236149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:27.252973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] The token cache is being invalidate because project 29b8b3474cd94a59a848ecb539898023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:27.253407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.254382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.256121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1050/2116] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:27 2022] DELETE /identity/v3/projects/29b8b3474cd94a59a848ecb539898023 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:27.260130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:27.270830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d5b6-5003-4e6a-b7e7-7859861392f8 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:27.339079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:27.339905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:27.340187 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:27.340460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] PATH_INFO: `/v3/users/b592083182c9470cac99e30a37481dda` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:27.341647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b592083182c9470cac99e30a37481dda)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:27.343566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:27.393006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b592083182c9470cac99e30a37481dda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.402062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b592083182c9470cac99e30a37481dda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.409560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b592083182c9470cac99e30a37481dda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.410225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b592083182c9470cac99e30a37481dda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.422536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b592083182c9470cac99e30a37481dda'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:27.430041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1067/2117] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:27 2022] DELETE /identity/v3/users/b592083182c9470cac99e30a37481dda => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:27.737638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:27.740426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:27.765758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dee14a-eb61-41b6-aa32-8d57d6660369 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:27.886462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:27.887376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:27.896054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:27.896676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] PATH_INFO: `/v3/users/0098866354f04715b5829a927660ab7a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:27.898276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0098866354f04715b5829a927660ab7a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:27.900479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:27.976655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:27.978768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.002787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695782f-beb2-4f9b-b366-c3ee2a6c5071 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.014098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.014742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.015029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.015305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] PATH_INFO: `/v3/projects/fa65e81a15624b3da74c9bfe483126fc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.018912 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa65e81a15624b3da74c9bfe483126fc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.020788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.021204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0098866354f04715b5829a927660ab7a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.034548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0098866354f04715b5829a927660ab7a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.036044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0098866354f04715b5829a927660ab7a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.047494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0098866354f04715b5829a927660ab7a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.075106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4a530718-3065-4e74-a64e-be7ae8d3c9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0098866354f04715b5829a927660ab7a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.083942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1051/2118] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:27 2022] DELETE /identity/v3/users/0098866354f04715b5829a927660ab7a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:28.088305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.088686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.088989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.097889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa65e81a15624b3da74c9bfe483126fc', 'request_id': 'req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.119485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa65e81a15624b3da74c9bfe483126fc', 'request_id': 'req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.124411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:28.126282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.127571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa65e81a15624b3da74c9bfe483126fc', 'request_id': 'req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.146626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] The token cache is being invalidate because project fa65e81a15624b3da74c9bfe483126fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:28.146932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.148743 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-223b24bc-ca11-4dc1-8bea-512fac7bb4b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.155245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1068/2119] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:27 2022] DELETE /identity/v3/projects/fa65e81a15624b3da74c9bfe483126fc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:28.159055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf0ff2-7703-472a-8358-f2b2d6253990 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.160573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf0ff2-7703-472a-8358-f2b2d6253990 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.160944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf0ff2-7703-472a-8358-f2b2d6253990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.169058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:28.190251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8ebf0ff2-7703-472a-8358-f2b2d6253990 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:28.193197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ebf0ff2-7703-472a-8358-f2b2d6253990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.240840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1052/2120] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:28.251860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.258267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.280687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:28.344045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1069/2121] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:28.348460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.348750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.349321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.371810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bf744-3e5d-449c-9a16-ac2cb9beca3d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.377390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:28.378666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.383350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.384019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.384303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.384580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.402981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1070/2122] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:28.403170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.403170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.404250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.418290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbc212-ff71-4e83-a237-9cb92f684607 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.419407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1053/2123] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:28.440704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.441195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.441964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.442131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.443226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:28.443226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.445825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.463828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1071/2124] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:28.470392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.484644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb5820-ea0b-4a57-97d7-732de171b2d8 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.491023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.492155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.492607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.492774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.494632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.498733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.536427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1054/2125] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:28.541479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.556284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afd0df-928c-49a6-bde8-3a9cab1f971b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.667879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.668503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.668785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.669191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.669824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.671868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.694639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1072/2126] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:28.700091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.710937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390654c-2958-4579-8123-bf4fd3355775 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.715680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.716318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.716605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.716867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.717917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.719175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.732720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1055/2127] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:28.737951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.749404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20662adc-7a82-40ce-bb7b-2d14324a6c6f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.753043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.754487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.755100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.755410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.755716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.756378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.756936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.758625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.771932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1073/2128] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:28.776724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.788690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98dc7-fca0-4492-b45e-b7df8bf5616c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.788903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5daa-7ff6-47c0-b606-def590a3333b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:28.793782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.794565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.795017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.795298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] PATH_INFO: `/v3/projects/e2da6c60426c43f485445c421dce1249` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.796266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc015eb3-1fb0-44c7-bf95-957e5a685a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:28.796676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2da6c60426c43f485445c421dce1249)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.797193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015eb3-1fb0-44c7-bf95-957e5a685a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.797659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015eb3-1fb0-44c7-bf95-957e5a685a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.798025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015eb3-1fb0-44c7-bf95-957e5a685a50 admin admin] PATH_INFO: `/v3/projects/9385d8348a3d4c7da663aa594ce09f36/users/4f8b106e81f044acad891fa75365c92c/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.798624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.818552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc015eb3-1fb0-44c7-bf95-957e5a685a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9385d8348a3d4c7da663aa594ce09f36, user_id=4f8b106e81f044acad891fa75365c92c, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:28.822908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc015eb3-1fb0-44c7-bf95-957e5a685a50 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:28.851246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1074/2129] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:28 2022] PUT /identity/v3/projects/9385d8348a3d4c7da663aa594ce09f36/users/4f8b106e81f044acad891fa75365c92c/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:28.862542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:28.863027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:28.863858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:28.878660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2da6c60426c43f485445c421dce1249', 'request_id': 'req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.889985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2da6c60426c43f485445c421dce1249', 'request_id': 'req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.897731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] MFA Rules not processed for user `4f8b106e81f044acad891fa75365c92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:28.899052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2da6c60426c43f485445c421dce1249', 'request_id': 'req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.899423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.914823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] The token cache is being invalidate because project e2da6c60426c43f485445c421dce1249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:28.915117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.915803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:28.917615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1056/2130] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:28 2022] DELETE /identity/v3/projects/e2da6c60426c43f485445c421dce1249 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:28.921583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.924370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:28.935299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:28.991267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1075/2131] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:28.996950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:28.998675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.005701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36666f8a-068b-48cc-bb62-cdbf4a7db9d5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.010870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.011515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.011863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.012143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] PATH_INFO: `/v3/users/16e8f2d9c877406aaf3508ac9a887c16` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.013363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e8f2d9c877406aaf3508ac9a887c16)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.015282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.015570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe77b81-bbf9-46cd-a14c-34097ce5aa16 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.021127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:29.022223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.022837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.023231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.023538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.024302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.027327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.067065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1076/2132] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:28 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:29.068057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.088803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e8f2d9c877406aaf3508ac9a887c16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.096803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e8f2d9c877406aaf3508ac9a887c16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.099394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e8f2d9c877406aaf3508ac9a887c16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.110627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e8f2d9c877406aaf3508ac9a887c16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.129439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e8f2d9c877406aaf3508ac9a887c16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.140209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1057/2133] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:28 2022] DELETE /identity/v3/users/16e8f2d9c877406aaf3508ac9a887c16 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:29.148774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0bdc6-9363-4760-b7a8-727ff4844399 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.236569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.237637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.238195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.238195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.239023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.241775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.266927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1077/2134] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:29 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:29.271516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.273377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.288618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23794e28-d809-43fd-8bd7-74221385a938 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.295763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.296709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.297288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.298253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.299197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.301160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.323829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1058/2135] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:29.328180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.341958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54d9d1-1ace-49a5-aacc-debfb9a375ef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.350505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.351176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.352426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.352598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] PATH_INFO: `/v3/projects/abe520a88e9c4eb3af1c54507b931554/users/d69d6cb4b3f04bc78cd7a5f58ea0388b/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.376193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe520a88e9c4eb3af1c54507b931554, user_id=d69d6cb4b3f04bc78cd7a5f58ea0388b, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.379009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.402179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1078/2136] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:29 2022] PUT /identity/v3/projects/abe520a88e9c4eb3af1c54507b931554/users/d69d6cb4b3f04bc78cd7a5f58ea0388b/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:29.403752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.416564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6f09-d09c-4835-9d8b-b290348a2d89 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.479630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.481403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.514448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.515120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.515329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.515756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.516417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.518749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.522581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:29.535635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1059/2137] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:29.542292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.553038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae98c10-3e3a-47e2-ad23-59d63733eccf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.559535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8d8c8-aec9-433f-b142-13e9b26fed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.560395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d8c8-aec9-433f-b142-13e9b26fed79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.560812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d8c8-aec9-433f-b142-13e9b26fed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.561093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d8c8-aec9-433f-b142-13e9b26fed79 admin admin] PATH_INFO: `/v3/projects/abe520a88e9c4eb3af1c54507b931554/users/d69d6cb4b3f04bc78cd7a5f58ea0388b/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.563059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d8c8-aec9-433f-b142-13e9b26fed79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe520a88e9c4eb3af1c54507b931554, user_id=d69d6cb4b3f04bc78cd7a5f58ea0388b, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.565946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d8c8-aec9-433f-b142-13e9b26fed79 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.614670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a413b00-e8a2-4165-9ae2-23b498d4aadd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.619738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1060/2138] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:29 2022] PUT /identity/v3/projects/abe520a88e9c4eb3af1c54507b931554/users/d69d6cb4b3f04bc78cd7a5f58ea0388b/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:29.631783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.632076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.632355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.667383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] MFA Rules not processed for user `d69d6cb4b3f04bc78cd7a5f58ea0388b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:29.668647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.693157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.693795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.694079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.694504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] PATH_INFO: `/v3/projects/4e7a559007464521b8527e0b73004e9e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.696365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e7a559007464521b8527e0b73004e9e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.698529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.728558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:29.751095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e7a559007464521b8527e0b73004e9e', 'request_id': 'req-3b623482-52f0-413d-a831-32cd99713282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.760048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e7a559007464521b8527e0b73004e9e', 'request_id': 'req-3b623482-52f0-413d-a831-32cd99713282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.762234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1061/2139] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:29.763671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.765265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.769828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e7a559007464521b8527e0b73004e9e', 'request_id': 'req-3b623482-52f0-413d-a831-32cd99713282'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.783168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] The token cache is being invalidate because project 4e7a559007464521b8527e0b73004e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:29.783461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.784220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.784901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f9b07-2b7e-4a04-892d-a4fc09efeb70 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.786118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1079/2140] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:29 2022] DELETE /identity/v3/projects/4e7a559007464521b8527e0b73004e9e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:29.786704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:29.788091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.850330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b623482-52f0-413d-a831-32cd99713282 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:29.856264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.856941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.857248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.857521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.858099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:29.859109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.859589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.859868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.860234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] PATH_INFO: `/v3/users/3e146e6da0294fa78e07d8c1e971af3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.861403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e146e6da0294fa78e07d8c1e971af3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.863515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.870745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:29.912142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e146e6da0294fa78e07d8c1e971af3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.919919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e146e6da0294fa78e07d8c1e971af3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.920607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e146e6da0294fa78e07d8c1e971af3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.928475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e146e6da0294fa78e07d8c1e971af3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.929445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:29.931768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7fc69-8fc4-44eb-82ca-2ee6ffd40f2f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:29.941650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1080/2141] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:29.942881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.943138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ef6a007-0b33-4fcd-99d6-b36ec7c15a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e146e6da0294fa78e07d8c1e971af3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:29.943260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.943428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.950539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1062/2142] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:29 2022] DELETE /identity/v3/users/3e146e6da0294fa78e07d8c1e971af3e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:29.965862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:29.967147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:29.968458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46545f-1649-498b-a1fb-4ed490cfba5a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:29.968752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46545f-1649-498b-a1fb-4ed490cfba5a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:29.969163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46545f-1649-498b-a1fb-4ed490cfba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:29.996758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e46545f-1649-498b-a1fb-4ed490cfba5a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:29.998057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e46545f-1649-498b-a1fb-4ed490cfba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.056654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1081/2143] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:30.059354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.062958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.080060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1063/2144] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:30.081424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.081791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.082134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.090608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:30.106123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:30.107403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.115652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1064/2145] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:30.116458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.117987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.130665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660efad-beb2-4224-af3e-d6dfa474a7b6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.134408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.135008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.135297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.135553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.137462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.176307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b7b3-3a56-43f8-8563-d6fed575bf83 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.180630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.181240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.181518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.181791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.191795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.193238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.203055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.203820 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1065/2146] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:30.204579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.204836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.210451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:30.218613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d8fee-71ac-4a8b-9c11-2dfd22939706 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.224065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.225484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.225827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.226103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.227425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:30.227758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.227990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1082/2147] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:30.229804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.245454 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1066/2148] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:30.249921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.265877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0a97e-f7c1-4959-8515-995a7c20b8d0 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.272199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.273957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.276861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.276861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.276861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.277363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.326848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1083/2149] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:30.334905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.344516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b10a8-3837-4ada-8b43-a77f71e415c2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.437520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.437520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.437520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.437520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.438018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.438746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.440484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.443355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.466654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4e915-acee-47f9-8680-e9b7adf0c227 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.477636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.478304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.478588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.478866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] PATH_INFO: `/v3/projects/8c78ed6f8b354b1b9b07151736df1386` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.482510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1067/2150] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:30.482663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c78ed6f8b354b1b9b07151736df1386)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.484866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.485712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.498168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44806a5-8383-41ca-9ee2-883085b47825 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.504396 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.505181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.505472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.505749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.506561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.509052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.525720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1068/2151] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:30.527071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.541305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f659d7-0a62-40ce-b9f8-2004301bcfe5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.556784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c78ed6f8b354b1b9b07151736df1386', 'request_id': 'req-601edd8a-de51-45c3-800d-9b239b04528e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:30.567661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c78ed6f8b354b1b9b07151736df1386', 'request_id': 'req-601edd8a-de51-45c3-800d-9b239b04528e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:30.580412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c78ed6f8b354b1b9b07151736df1386', 'request_id': 'req-601edd8a-de51-45c3-800d-9b239b04528e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:30.601549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] The token cache is being invalidate because project 8c78ed6f8b354b1b9b07151736df1386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:30.602505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:30.603699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601edd8a-de51-45c3-800d-9b239b04528e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:30.606843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1084/2152] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:30 2022] DELETE /identity/v3/projects/8c78ed6f8b354b1b9b07151736df1386 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:30.613290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179ae28-a3d6-4693-bc37-35dea9b0320c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.614840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179ae28-a3d6-4693-bc37-35dea9b0320c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.615122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179ae28-a3d6-4693-bc37-35dea9b0320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.647068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c179ae28-a3d6-4693-bc37-35dea9b0320c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:30.648911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c179ae28-a3d6-4693-bc37-35dea9b0320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.654691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330400-8365-4e88-8728-65650cda95be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.656587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330400-8365-4e88-8728-65650cda95be admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.657271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1085/2153] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:30.657619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330400-8365-4e88-8728-65650cda95be admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.658015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330400-8365-4e88-8728-65650cda95be admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.659963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330400-8365-4e88-8728-65650cda95be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.664119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330400-8365-4e88-8728-65650cda95be admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.665702 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.666391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.666634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.676865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:30.692047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1069/2154] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:30.701764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330400-8365-4e88-8728-65650cda95be admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.703552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15330400-8365-4e88-8728-65650cda95be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.704072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:30.705586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.716875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330400-8365-4e88-8728-65650cda95be admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.721841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172390f-d870-46d8-8927-00252c9da5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.722676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172390f-d870-46d8-8927-00252c9da5cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.723643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172390f-d870-46d8-8927-00252c9da5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.723643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172390f-d870-46d8-8927-00252c9da5cb admin admin] PATH_INFO: `/v3/projects/ac38dd3fe6dd4991b3b3c9610d7e82f5/users/a02d8582f5d04959ab977193bd53e947/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.723839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1086/2155] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:30.731241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.745565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172390f-d870-46d8-8927-00252c9da5cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac38dd3fe6dd4991b3b3c9610d7e82f5, user_id=a02d8582f5d04959ab977193bd53e947, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.748486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172390f-d870-46d8-8927-00252c9da5cb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.753499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adfade-1848-43cc-b976-0deaef2d5ef4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.759402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.759916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.760139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.760412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.761089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:30.761340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.763174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.764815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1070/2156] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:30 2022] PUT /identity/v3/projects/ac38dd3fe6dd4991b3b3c9610d7e82f5/users/a02d8582f5d04959ab977193bd53e947/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:30.775932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.776241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.776478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.783653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1087/2157] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:30.785904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.797070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71b1f3-0431-400d-b10f-448fa566dbee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.815784 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] MFA Rules not processed for user `a02d8582f5d04959ab977193bd53e947`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:30.817660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:30.883035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:30.884078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:30.884078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:30.884713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:30.888084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:30.890002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:30.926122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1071/2158] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:30.931102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.947803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1088/2159] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:30.950776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373a344-0c3b-4701-abd2-8a9932a13c68 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:30.952154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:30.995179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:31.043690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.044312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.044746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.045682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.053506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.056521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.091294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cfcbd-7d61-4c1c-a753-82a72c654823 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.109697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1072/2160] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:31.113990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.116665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.122454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.122749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.123030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.123706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.125830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.154486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1089/2161] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:30 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:31.166543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.178465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf704d42-b18d-42a9-b60c-93dd928fe4d1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.178637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:31.195697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d70843-f774-4ce8-97ec-0b4441edea3a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.202060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.202707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.203015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.203287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.204114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.206022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.228463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1073/2162] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:31 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:31.232658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.242926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54507e-5b8b-4d5f-8f03-96e70f4d3902 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.249379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.249994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.250273 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.250541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.251205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.252888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.260867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.261611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.262026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.262370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.263029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.264865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.265573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1074/2163] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:31.269179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.277317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1090/2164] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:31.281230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.281466 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86714c3b-24f9-472b-a0c5-3e7ec6119f01 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.287442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.288201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.288514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.289121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.289965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.291888 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.292572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be0ca7-8826-4661-a062-53bfccfd6469 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.301686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.302624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.303027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.303410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.303816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1075/2165] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:31.304292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.306800 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.309392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.323832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1091/2166] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:31.325521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.326860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242e97-ff1a-4fa1-8e65-d5bc555c2b23 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.332070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd70a8d-44c9-48a1-bb2f-ace1e0c46c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.333379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70a8d-44c9-48a1-bb2f-ace1e0c46c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.333676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70a8d-44c9-48a1-bb2f-ace1e0c46c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.333959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70a8d-44c9-48a1-bb2f-ace1e0c46c57 admin admin] PATH_INFO: `/v3/projects/1328a0d57ea94d848471f68b33aafd60/users/fc877b1098344da3bec48d1693181c19/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.339196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97d2e8-1a2f-4a2e-8495-a5c9a92b441b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.345414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75b95d-76e6-4c86-98be-8f3816439ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.346508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75b95d-76e6-4c86-98be-8f3816439ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.346967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75b95d-76e6-4c86-98be-8f3816439ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.347764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75b95d-76e6-4c86-98be-8f3816439ca4 admin admin] PATH_INFO: `/v3/projects/a26ace4e3d0747f6b5dc8687003a2e34/users/e90017290ec545a088568cd802c62d75/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.350224 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd70a8d-44c9-48a1-bb2f-ace1e0c46c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1328a0d57ea94d848471f68b33aafd60, user_id=fc877b1098344da3bec48d1693181c19, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.353153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd70a8d-44c9-48a1-bb2f-ace1e0c46c57 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.366889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75b95d-76e6-4c86-98be-8f3816439ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26ace4e3d0747f6b5dc8687003a2e34, user_id=e90017290ec545a088568cd802c62d75, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.367900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1076/2167] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:31 2022] PUT /identity/v3/projects/1328a0d57ea94d848471f68b33aafd60/users/fc877b1098344da3bec48d1693181c19/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:31.369790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75b95d-76e6-4c86-98be-8f3816439ca4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.383080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.383450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.383739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.396808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1092/2168] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:31 2022] PUT /identity/v3/projects/a26ace4e3d0747f6b5dc8687003a2e34/users/e90017290ec545a088568cd802c62d75/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:31.436601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] MFA Rules not processed for user `fc877b1098344da3bec48d1693181c19`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:31.437957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:31.461897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.462395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.462970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.491220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] MFA Rules not processed for user `e90017290ec545a088568cd802c62d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:31.492793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:31.505869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1077/2169] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:31.510873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.556935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:31.564360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1093/2170] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:31.572629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.574013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:31.574241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303997bb-3e95-44c4-98f6-3a1677a41abc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.579007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.579613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.580328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.580328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.580813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.582743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.615814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1078/2171] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:31 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:04:31.616777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.627426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:31.636614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbe2a0-c3f9-4114-ae87-9dfd8282f180 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.681065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c76-dafa-4ac5-a259-0182f90aa4f3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.685855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.686019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.686474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.686613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.686756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.686895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.687147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.687280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.687788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.689695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.716815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1079/2172] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:31 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:31.718095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.728735 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70843d-28a4-481e-b1ee-3ea6f6ddb9ae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.733587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.734184 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.734464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.734730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.735525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.737402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.750300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1080/2173] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:31.755089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.755221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.756753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.766277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ed411-952e-47e9-92e5-a520769e395e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.767321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1094/2174] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:04:31.771002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.771681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.771961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.772235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] PATH_INFO: `/v3/projects/2a5cae2d172f4d27b7f52ef94bb0ca95/users/ed51851fe9c24e87b72b3065cf158833/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.787819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5cae2d172f4d27b7f52ef94bb0ca95, user_id=ed51851fe9c24e87b72b3065cf158833, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.791102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.804283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1081/2175] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:31 2022] PUT /identity/v3/projects/2a5cae2d172f4d27b7f52ef94bb0ca95/users/ed51851fe9c24e87b72b3065cf158833/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:31.809378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.819724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975909-79d4-4809-ade6-2ff93b1d6abf service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.889653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e3e62-75bd-4f9d-859b-fdcd34641703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.890324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e3e62-75bd-4f9d-859b-fdcd34641703 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.890606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e3e62-75bd-4f9d-859b-fdcd34641703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.890880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e3e62-75bd-4f9d-859b-fdcd34641703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.891521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e3e62-75bd-4f9d-859b-fdcd34641703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.893173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e3e62-75bd-4f9d-859b-fdcd34641703 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.906928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1095/2176] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:31.908326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:31.920018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3607cd-9ae8-4153-b05b-0ec223ace615 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:31.925065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:31.925963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.926344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.926711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] PATH_INFO: `/v3/projects/2a5cae2d172f4d27b7f52ef94bb0ca95/users/ed51851fe9c24e87b72b3065cf158833/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.928922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5cae2d172f4d27b7f52ef94bb0ca95, user_id=ed51851fe9c24e87b72b3065cf158833, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:31.932765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:31.943695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1082/2177] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:31 2022] PUT /identity/v3/projects/2a5cae2d172f4d27b7f52ef94bb0ca95/users/ed51851fe9c24e87b72b3065cf158833/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:31.954519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887302-3230-4475-8c34-00187bec6aec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:31.954811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887302-3230-4475-8c34-00187bec6aec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:31.955112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887302-3230-4475-8c34-00187bec6aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:31.969494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:31.985117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ae887302-3230-4475-8c34-00187bec6aec None None] MFA Rules not processed for user `ed51851fe9c24e87b72b3065cf158833`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:31.986348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae887302-3230-4475-8c34-00187bec6aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:32.062593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1096/2178] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:32.073925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:32.147102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e1819-1d20-45d1-ba94-6d9df82ad2ee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:32.155775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:32.156004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:32.156280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:32.156549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:32.224225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:32.225752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:32.235890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1083/2179] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:34.260832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:34.261374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:34.261480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:34.281909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:34.283129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:34.343720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1097/2180] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:04:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:34.357245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:34.358846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:34.365678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:34.453774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ade81-6ccc-475f-b7dc-55967962485b service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:34.458805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:34.459421 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:34.459729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:34.460007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:34.469403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:34.471309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:34.481688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1084/2181] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:04:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:04:34.876112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:34.877525 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:34.889475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2f380-74ae-4f84-8a3a-084e09c478e6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:34.893162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:34.893775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:34.894054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:34.894463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:34.969232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:34.970823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:34.981464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1098/2182] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:36.783174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:36.793150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60da8e5-9f99-4911-b889-01a58ee26834 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:36.796948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c0b5e-fc49-4d02-bb22-b16e1d93638a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:36.797681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0b5e-fc49-4d02-bb22-b16e1d93638a service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:36.797967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0b5e-fc49-4d02-bb22-b16e1d93638a service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:36.798065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0b5e-fc49-4d02-bb22-b16e1d93638a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:36.808630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c0b5e-fc49-4d02-bb22-b16e1d93638a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:36.810053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c0b5e-fc49-4d02-bb22-b16e1d93638a service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:36.820374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1085/2183] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:38.633203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:38.635413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:38.745726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:38.801551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96133251-b4e6-4f47-bffe-f7e0772add1e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:38.816284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:38.817385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:38.818512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:38.819076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] PATH_INFO: `/v3/users/4f8b106e81f044acad891fa75365c92c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:38.821459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8b106e81f044acad891fa75365c92c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:38.824132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:38.942349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8b106e81f044acad891fa75365c92c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:38.963374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8b106e81f044acad891fa75365c92c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:38.977762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8b106e81f044acad891fa75365c92c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:38.978708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8b106e81f044acad891fa75365c92c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:39.014700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8b106e81f044acad891fa75365c92c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:39.046477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1099/2184] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:38 2022] DELETE /identity/v3/users/4f8b106e81f044acad891fa75365c92c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:39.051723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:39.052887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:39.053762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:39.104351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:39.105695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:39.204812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1086/2185] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:39.217468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:39.219403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:39.323301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90307f-109b-46a6-9f9e-6b575205808f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:39.328450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:39.329507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:39.330061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:39.330527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:39.351070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:39.353765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:39.373885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1100/2186] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:39.845952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:39.848021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:39.896693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5be0c-f482-432a-96df-9ad3515de760 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:39.903494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:39.904445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:39.904934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:39.905463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] PATH_INFO: `/v3/projects/9385d8348a3d4c7da663aa594ce09f36` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:39.908494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9385d8348a3d4c7da663aa594ce09f36)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:39.911277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:40.014237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9385d8348a3d4c7da663aa594ce09f36', 'request_id': 'req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.023445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9385d8348a3d4c7da663aa594ce09f36', 'request_id': 'req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.033860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9385d8348a3d4c7da663aa594ce09f36', 'request_id': 'req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.067403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] The token cache is being invalidate because project 9385d8348a3d4c7da663aa594ce09f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:40.068020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.068403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.070690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1087/2187] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:39 2022] DELETE /identity/v3/projects/9385d8348a3d4c7da663aa594ce09f36 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:40.074962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:40.085476 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafe6e-0586-42e8-9114-7a1718f2b883 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:40.147181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:40.155715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:40.156398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.156975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.157713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] PATH_INFO: `/v3/users/d69d6cb4b3f04bc78cd7a5f58ea0388b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.159140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d69d6cb4b3f04bc78cd7a5f58ea0388b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:40.161622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:40.219176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd69d6cb4b3f04bc78cd7a5f58ea0388b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.228000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd69d6cb4b3f04bc78cd7a5f58ea0388b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.235273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd69d6cb4b3f04bc78cd7a5f58ea0388b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.235963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd69d6cb4b3f04bc78cd7a5f58ea0388b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.248998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba4ba831-2ebd-4d77-b052-692e5ade7afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd69d6cb4b3f04bc78cd7a5f58ea0388b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.255122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1101/2188] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:40 2022] DELETE /identity/v3/users/d69d6cb4b3f04bc78cd7a5f58ea0388b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:40.519852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:40.521405 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:40.531425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:40.535769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca924-0529-495e-aafa-15fe2b1a9ad3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:40.597635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:40.598262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.598538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.598808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] PATH_INFO: `/v3/projects/abe520a88e9c4eb3af1c54507b931554` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.599934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe520a88e9c4eb3af1c54507b931554)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:40.602814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:40.649808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe520a88e9c4eb3af1c54507b931554', 'request_id': 'req-e98e1f31-a71b-4c29-9710-24fcdc0848c1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.655083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe520a88e9c4eb3af1c54507b931554', 'request_id': 'req-e98e1f31-a71b-4c29-9710-24fcdc0848c1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.661085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe520a88e9c4eb3af1c54507b931554', 'request_id': 'req-e98e1f31-a71b-4c29-9710-24fcdc0848c1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.674185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] The token cache is being invalidate because project abe520a88e9c4eb3af1c54507b931554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:40.674470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.675109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e98e1f31-a71b-4c29-9710-24fcdc0848c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:40.676627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1088/2189] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:40 2022] DELETE /identity/v3/projects/abe520a88e9c4eb3af1c54507b931554 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:40.681707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.682003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.682281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.703572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:40.704850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:40.765916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1102/2190] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:40.774364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.774672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.774943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.794857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:40.796144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:40.804293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1089/2191] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:40.808522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:40.818233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1158df-3a97-4972-b606-f57e65ff4558 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:40.823014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:40.823626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.823915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.824190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.824832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:40.825134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:40.826727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:40.841726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1103/2192] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:40.843090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:40.853323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9191483-79ea-45e4-99cc-89842596ff88 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:40.857973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:40.858574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.858851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.859233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.859897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:40.861792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:40.888825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1090/2193] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:40 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:40.893072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:40.903502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb944983-3de1-4918-81d0-2bbba1a8b626 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:40.958120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:40.958725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:40.959007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:40.959282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:40.959932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:40.962343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:40.971958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:40.984689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1104/2194] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:40 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:40.988687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:40.999428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d304f-1529-4eb4-a738-e8caf6990a55 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.004463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.005070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.005352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.005628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.006278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.008496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:41.018865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1091/2195] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:41.023477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:41.034304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fac254-9926-4eab-9ffe-845a07355d9b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.040121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85737733-c0f7-4c13-bec1-740aa3ae115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.040735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737733-c0f7-4c13-bec1-740aa3ae115c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.041036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737733-c0f7-4c13-bec1-740aa3ae115c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.041312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737733-c0f7-4c13-bec1-740aa3ae115c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.041970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85737733-c0f7-4c13-bec1-740aa3ae115c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.043580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85737733-c0f7-4c13-bec1-740aa3ae115c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:41.056853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1105/2196] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:41.060840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:41.074557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3fe17-968d-45f5-83a9-e9c397e99e24 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.079716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.080390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.080995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.081299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] PATH_INFO: `/v3/projects/0156f3cac80c4983be0086e124c57dbf/users/2a4e18da56724a309a4b69accb5685a2/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.102457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0156f3cac80c4983be0086e124c57dbf, user_id=2a4e18da56724a309a4b69accb5685a2, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.105368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:41.133214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1092/2197] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:41 2022] PUT /identity/v3/projects/0156f3cac80c4983be0086e124c57dbf/users/2a4e18da56724a309a4b69accb5685a2/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:41.142522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.143044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.143141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.173516 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] MFA Rules not processed for user `2a4e18da56724a309a4b69accb5685a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:41.175295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:41.253723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1106/2198] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:41.265721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:41.267423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:41.301156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:41.327501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dba6c-47d9-40dc-a070-3b4798d1ca8d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.331142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91a541-bbd3-4a23-9110-51e8522fb3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.331791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91a541-bbd3-4a23-9110-51e8522fb3df service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.332073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91a541-bbd3-4a23-9110-51e8522fb3df service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.332345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91a541-bbd3-4a23-9110-51e8522fb3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.341050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91a541-bbd3-4a23-9110-51e8522fb3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.342465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91a541-bbd3-4a23-9110-51e8522fb3df service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:41.351126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1093/2199] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:41.372778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:41.435086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0eeae-dea0-4234-a41e-29e6c37caf04 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.439665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.440293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.440573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.440847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] PATH_INFO: `/v3/users/2a4e18da56724a309a4b69accb5685a2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.441967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4e18da56724a309a4b69accb5685a2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.443910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:41.488844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4e18da56724a309a4b69accb5685a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:41.496617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4e18da56724a309a4b69accb5685a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:41.503314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4e18da56724a309a4b69accb5685a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:41.504214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4e18da56724a309a4b69accb5685a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:41.512850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:41.517869 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4e18da56724a309a4b69accb5685a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:41.525343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1107/2200] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:41 2022] DELETE /identity/v3/users/2a4e18da56724a309a4b69accb5685a2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:41.529887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.530177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.530455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.549363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:41.550554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:41.611386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1094/2201] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:41.622177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:41.628945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:41.680571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c7b3-9ff0-4868-aec6-9283a339a090 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.684130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870a9bd-ccee-468a-8b98-4ccaf54c983a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.684726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870a9bd-ccee-468a-8b98-4ccaf54c983a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.685003 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870a9bd-ccee-468a-8b98-4ccaf54c983a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.685272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870a9bd-ccee-468a-8b98-4ccaf54c983a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.694098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870a9bd-ccee-468a-8b98-4ccaf54c983a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.696297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870a9bd-ccee-468a-8b98-4ccaf54c983a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:41.705721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1108/2202] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:41.968207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:41.969680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:41.985014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb9c78-b2aa-48f9-819d-756e78a6e3cb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:41.989741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:41.990391 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:41.990651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:41.991026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] PATH_INFO: `/v3/projects/0156f3cac80c4983be0086e124c57dbf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:41.992914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0156f3cac80c4983be0086e124c57dbf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:41.995107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.047178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0156f3cac80c4983be0086e124c57dbf', 'request_id': 'req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:42.053549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0156f3cac80c4983be0086e124c57dbf', 'request_id': 'req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:42.061204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0156f3cac80c4983be0086e124c57dbf', 'request_id': 'req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:42.076218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] The token cache is being invalidate because project 0156f3cac80c4983be0086e124c57dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:42.076846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:42.077794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cae5329-4c6a-4307-a5d9-b6f0e753aeeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:42.079622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1095/2203] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:41 2022] DELETE /identity/v3/projects/0156f3cac80c4983be0086e124c57dbf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:42.086365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.086906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.087253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.109591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:42.110910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:42.208782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1109/2204] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:42.216919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.217550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.217986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.260641 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:42.264135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:42.314410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1096/2205] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:42.323090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:42.340138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8563a2-effc-438d-9016-d631d8c8edc2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:42.345571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:42.346188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.346471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.346744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.347475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:42.347817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:42.349948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.364885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1110/2206] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:42.370557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:42.384849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac0738-be81-4919-ae5c-c5d35ecedf5a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:42.398695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:42.399453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.399826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.400169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.400865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:42.402917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.457234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1097/2207] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:42 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:42.463275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:42.477745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491f94-b303-45a5-8e99-da273747e110 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:42.570893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:42.571745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.572347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.572727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.573990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:42.577256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.611089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1111/2208] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:42 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:42.615258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:42.631171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b09da-a71e-43aa-b057-f328ee49af0f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:42.641307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:42.642577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.642994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.643330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.644395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:42.646374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.664339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1098/2209] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:42.669145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:42.682932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18fb9e-db9a-4a3d-9c5b-d41d98928dc9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:42.697594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443cd2-bea5-42df-972b-1ce9f299d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:42.699058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443cd2-bea5-42df-972b-1ce9f299d20b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.700842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443cd2-bea5-42df-972b-1ce9f299d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.701219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443cd2-bea5-42df-972b-1ce9f299d20b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.703311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443cd2-bea5-42df-972b-1ce9f299d20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:42.709462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443cd2-bea5-42df-972b-1ce9f299d20b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.719052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:42.731674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1112/2210] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:42.736234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:42.769423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0f502-cb60-4468-9a56-26c2460dc1d5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:42.786300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:42.787099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.787312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.787587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] PATH_INFO: `/v3/projects/db25f0cca74b49ae954f1a057ce96d39/users/53cd10ddd87f4f608589bc67f32ab62d/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:42.851386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db25f0cca74b49ae954f1a057ce96d39, user_id=53cd10ddd87f4f608589bc67f32ab62d, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:42.868619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:42.881127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:42.918437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1099/2211] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:42 2022] PUT /identity/v3/projects/db25f0cca74b49ae954f1a057ce96d39/users/53cd10ddd87f4f608589bc67f32ab62d/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:42.931069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:42.932565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:42.933179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:43.009985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] MFA Rules not processed for user `53cd10ddd87f4f608589bc67f32ab62d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:43.011504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:43.187942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1113/2212] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:43.189426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:43.349093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f954d-d864-4300-8505-44d2c16d8ec0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:43.361686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:43.362306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:43.362599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:43.374143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:43.374931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:43.377323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:43.435353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1100/2213] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:43 2022] POST /identity/v3/projects => generated 373 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:43.436504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:43.458872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c72115-3a34-4839-bdcd-32ff25df22b2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:43.647096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:43.650520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:43.651091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:43.651543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:43.652325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:43.664538 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:43.734871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1114/2214] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:43 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:43.738900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:43.761896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849e706-ea57-400d-bcba-d8afe067d93f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:43.783469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:43.784586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:43.784885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:43.785175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:43.785900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:43.787708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:43.836104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1101/2215] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:43.847693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:43.882075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e39ce-98d7-4624-bedb-ec013858b9a0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:43.885966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:43.887072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:43.887740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:43.888020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] PATH_INFO: `/v3/projects/6952bed394c543d79031bce6a0b16884/users/4bfa2fe10e57487fabe69776445cd1d5/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:43.907442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6952bed394c543d79031bce6a0b16884, user_id=4bfa2fe10e57487fabe69776445cd1d5, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:43.911023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:43.989826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1115/2216] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:43 2022] PUT /identity/v3/projects/6952bed394c543d79031bce6a0b16884/users/4bfa2fe10e57487fabe69776445cd1d5/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:43.995748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:44.034783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568375f7-ebc7-4f98-b1ba-68db5e59b460 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:44.185129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d114-ca99-4778-824f-ab2cf5093180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:44.186155 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d114-ca99-4778-824f-ab2cf5093180 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:44.186750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d114-ca99-4778-824f-ab2cf5093180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:44.187289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d114-ca99-4778-824f-ab2cf5093180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:44.188358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d114-ca99-4778-824f-ab2cf5093180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:44.190195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d114-ca99-4778-824f-ab2cf5093180 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:44.205291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1102/2217] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:44.209645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:44.223032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3546be4-76d0-4e49-a19e-31206bf68837 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:44.237303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:44.238125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:44.238478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:44.238827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] PATH_INFO: `/v3/projects/6952bed394c543d79031bce6a0b16884/users/4bfa2fe10e57487fabe69776445cd1d5/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:44.242623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6952bed394c543d79031bce6a0b16884, user_id=4bfa2fe10e57487fabe69776445cd1d5, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:44.245884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:44.268045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1116/2218] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:44 2022] PUT /identity/v3/projects/6952bed394c543d79031bce6a0b16884/users/4bfa2fe10e57487fabe69776445cd1d5/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:44.275304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb45f-f1c7-4c6f-87d7-be17c8e3084e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:44.275771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb45f-f1c7-4c6f-87d7-be17c8e3084e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:44.276804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb45f-f1c7-4c6f-87d7-be17c8e3084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:44.311280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-880fb45f-f1c7-4c6f-87d7-be17c8e3084e None None] MFA Rules not processed for user `4bfa2fe10e57487fabe69776445cd1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:44.313490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-880fb45f-f1c7-4c6f-87d7-be17c8e3084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:44.440659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:44.468654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1103/2219] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:44.481316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:44.483047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:44.564351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:44.591262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4fd08-5a23-41b0-bd88-5e12d13eb1eb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:44.598960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:44.601135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:44.601826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:44.602703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:44.704712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:44.714795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:44.743781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1117/2220] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:45.226321 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d04e-19e8-444f-b386-350c7518faec None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:45.226826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d04e-19e8-444f-b386-350c7518faec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:45.228468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d04e-19e8-444f-b386-350c7518faec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:45.294226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2067d04e-19e8-444f-b386-350c7518faec None None] MFA Rules not processed for user `cc885026afe041cca086367bb67d8a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:45.295674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2067d04e-19e8-444f-b386-350c7518faec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:45.411240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1104/2221] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:45.504950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:45.508045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:45.623799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fba33-3844-48aa-a3be-22368bd25a65 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:45.628277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:45.628906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:45.629446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:45.629716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] PATH_INFO: `/v3/users/a02d8582f5d04959ab977193bd53e947` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:45.630837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02d8582f5d04959ab977193bd53e947)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:45.633419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:45.685509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02d8582f5d04959ab977193bd53e947'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:45.693579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02d8582f5d04959ab977193bd53e947'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:45.706372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02d8582f5d04959ab977193bd53e947'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:45.707088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02d8582f5d04959ab977193bd53e947'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:45.719749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02d8582f5d04959ab977193bd53e947'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:45.726156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1118/2222] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:45 2022] DELETE /identity/v3/users/a02d8582f5d04959ab977193bd53e947 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:45.730716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:45.731003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:45.731276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:45.751626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:45.752754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:45.824467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1105/2223] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:45.832011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:45.833544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:45.859690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:45.903022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb3fa-73f6-48da-aac9-15c1b80bd221 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:45.906755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6a0cf-5ed4-4e79-94d4-b51d4206b1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:45.907370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6a0cf-5ed4-4e79-94d4-b51d4206b1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:45.907712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6a0cf-5ed4-4e79-94d4-b51d4206b1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:45.907990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6a0cf-5ed4-4e79-94d4-b51d4206b1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:45.919525 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6a0cf-5ed4-4e79-94d4-b51d4206b1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:45.921005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6a0cf-5ed4-4e79-94d4-b51d4206b1f3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:45.931006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1119/2224] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:46.206293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:46.208190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:46.226818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:46.227751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:46.228101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:46.237267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764e0e-4c8a-4f48-baa9-d376966a77db None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:46.245339 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:46.246068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:46.246415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:46.246828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] PATH_INFO: `/v3/projects/ac38dd3fe6dd4991b3b3c9610d7e82f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:46.248117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac38dd3fe6dd4991b3b3c9610d7e82f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:46.250102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:46.250713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] MFA Rules not processed for user `19da3c426c6e4c4a889ab5017804c92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:46.252331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:46.322361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac38dd3fe6dd4991b3b3c9610d7e82f5', 'request_id': 'req-899be078-231a-4cd4-90c0-484ce6e8603e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.328925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac38dd3fe6dd4991b3b3c9610d7e82f5', 'request_id': 'req-899be078-231a-4cd4-90c0-484ce6e8603e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.335741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1120/2225] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:46.336414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac38dd3fe6dd4991b3b3c9610d7e82f5', 'request_id': 'req-899be078-231a-4cd4-90c0-484ce6e8603e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.351622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] The token cache is being invalidate because project ac38dd3fe6dd4991b3b3c9610d7e82f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:46.351997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.352699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.354437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1106/2226] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:46 2022] DELETE /identity/v3/projects/ac38dd3fe6dd4991b3b3c9610d7e82f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:46.358575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:46.371580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db154aa1-7a9c-4c23-a629-eba637e4bc19 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:46.480899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:46.481552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:46.481904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:46.482179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] PATH_INFO: `/v3/users/fc877b1098344da3bec48d1693181c19` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:46.483322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc877b1098344da3bec48d1693181c19)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:46.485445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:46.556852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc877b1098344da3bec48d1693181c19'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.567182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc877b1098344da3bec48d1693181c19'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.577704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc877b1098344da3bec48d1693181c19'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.578561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc877b1098344da3bec48d1693181c19'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.580780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:46.595407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc877b1098344da3bec48d1693181c19'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:46.602407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1121/2227] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:46 2022] DELETE /identity/v3/users/fc877b1098344da3bec48d1693181c19 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:04:46.966272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:46.969359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:46.998653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be078-231a-4cd4-90c0-484ce6e8603e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:47.044923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:47.125074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:47.126453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:47.127452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:47.128698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] PATH_INFO: `/v3/projects/1328a0d57ea94d848471f68b33aafd60` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:47.131510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1328a0d57ea94d848471f68b33aafd60)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:47.134042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:47.199037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1328a0d57ea94d848471f68b33aafd60', 'request_id': 'req-540ff61b-f7ca-4b05-9603-837fb6f38e75'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.211293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1328a0d57ea94d848471f68b33aafd60', 'request_id': 'req-540ff61b-f7ca-4b05-9603-837fb6f38e75'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.219754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1328a0d57ea94d848471f68b33aafd60', 'request_id': 'req-540ff61b-f7ca-4b05-9603-837fb6f38e75'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.238349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] The token cache is being invalidate because project 1328a0d57ea94d848471f68b33aafd60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:47.238877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.239424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.242204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1107/2228] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:46 2022] DELETE /identity/v3/projects/1328a0d57ea94d848471f68b33aafd60 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:47.246367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:47.247963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:47.266486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccc0de-8bc5-4955-a758-1b9f957749ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:47.392712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:47.392712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:47.392712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:47.392712 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] PATH_INFO: `/v3/users/ed51851fe9c24e87b72b3065cf158833` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:47.393488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed51851fe9c24e87b72b3065cf158833)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:47.393655 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:47.481081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed51851fe9c24e87b72b3065cf158833'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.497226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed51851fe9c24e87b72b3065cf158833'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.512290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed51851fe9c24e87b72b3065cf158833'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.513569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed51851fe9c24e87b72b3065cf158833'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.531565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-94dd2e34-f125-4e14-83b2-7d3845cb3aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed51851fe9c24e87b72b3065cf158833'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:47.549803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1122/2229] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:47 2022] DELETE /identity/v3/users/ed51851fe9c24e87b72b3065cf158833 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:48.614543 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:48.618752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:48.643816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ff61b-f7ca-4b05-9603-837fb6f38e75 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:48.839863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:48.840570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:48.841089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:48.841516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] PATH_INFO: `/v3/projects/2a5cae2d172f4d27b7f52ef94bb0ca95` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:48.844368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5cae2d172f4d27b7f52ef94bb0ca95)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:48.846630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:48.977998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5cae2d172f4d27b7f52ef94bb0ca95', 'request_id': 'req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:48.981536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:49.012793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5cae2d172f4d27b7f52ef94bb0ca95', 'request_id': 'req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.031532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5cae2d172f4d27b7f52ef94bb0ca95', 'request_id': 'req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.057030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] The token cache is being invalidate because project 2a5cae2d172f4d27b7f52ef94bb0ca95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:49.057030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.057030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-82d7b22a-7267-4c7c-8ffc-28cf41f6681c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.062535 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1108/2230] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:48 2022] DELETE /identity/v3/projects/2a5cae2d172f4d27b7f52ef94bb0ca95 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:49.063215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:49.063676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:49.063805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:49.113937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:49.115627 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:49.195714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1123/2231] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:49.206632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:49.207258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:49.207818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:49.247386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:49.249750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:49.254415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:49.256521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:49.280506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1109/2232] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:49.285430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:49.287245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:49.301418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a337246-661e-4fe7-a449-a070f3e792b7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:49.313013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db405676-e342-454d-b1bd-7d2faab02cf1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:49.316679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:49.317774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:49.318050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:49.318425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] PATH_INFO: `/v3/users/53cd10ddd87f4f608589bc67f32ab62d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:49.320931 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53cd10ddd87f4f608589bc67f32ab62d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:49.327505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:49.327822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:49.328463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:49.328749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:49.329024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:49.332458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:04:49.332792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:49.338243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:49.365141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1110/2233] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:04:49.371758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:49.393316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d418744-0a68-41c7-b3bb-2b77acbfcd00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:49.423066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:49.423719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:49.424025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:49.424305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:49.424964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:49.437137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53cd10ddd87f4f608589bc67f32ab62d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.440156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:49.455017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53cd10ddd87f4f608589bc67f32ab62d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.476145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53cd10ddd87f4f608589bc67f32ab62d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.479590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53cd10ddd87f4f608589bc67f32ab62d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.498004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1111/2234] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:49 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:49.503509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:49.528385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:49.561528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b7b58a8-559a-4199-910f-6705b593bdb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53cd10ddd87f4f608589bc67f32ab62d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:49.595459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1124/2235] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:49 2022] DELETE /identity/v3/users/53cd10ddd87f4f608589bc67f32ab62d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:49.598291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:49.599917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:49.600220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:49.680274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:49.704404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:49.846496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d47998-0551-4fbb-99c4-e275d9049ccf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:49.981709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1125/2236] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:49.996720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:49.998175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:50.016093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.016859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.017241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.017879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.018775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.021987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.086451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1112/2237] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:49 2022] POST /identity/v3/users => generated 293 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:50.087486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:50.124299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbe9df-ed5e-40e1-9145-826998ba7ad5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:50.150678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.151400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.151737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.152032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.152690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.154355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.154936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d40dd4-45b4-4a40-9a74-e8887e1c3d8e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:50.162553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.163917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.164437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.164932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.179398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.182692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.188674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1113/2238] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:50.189614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:50.201053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1126/2239] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:50.221639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650930e-291e-42d6-ac78-617ed89711f1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:50.240754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e7b9-6cfd-4c2c-8e50-30cdb17e4d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.242265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e7b9-6cfd-4c2c-8e50-30cdb17e4d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.243622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e7b9-6cfd-4c2c-8e50-30cdb17e4d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.244698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e7b9-6cfd-4c2c-8e50-30cdb17e4d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.245536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e7b9-6cfd-4c2c-8e50-30cdb17e4d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.250359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e7b9-6cfd-4c2c-8e50-30cdb17e4d34 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.288134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1114/2240] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:50.294052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:50.308370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fe3b1-33e9-46e4-a6e8-e5eb1361c1ab service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:50.316402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.317380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.317668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.318098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] PATH_INFO: `/v3/projects/c5534652a6ac48d6b371756c9197d207/users/13cc07e4481b47ad8add86a3e475a93c/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.341482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5534652a6ac48d6b371756c9197d207, user_id=13cc07e4481b47ad8add86a3e475a93c, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.348252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.352380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:50.378411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1127/2241] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:50 2022] PUT /identity/v3/projects/c5534652a6ac48d6b371756c9197d207/users/13cc07e4481b47ad8add86a3e475a93c/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:50.390889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.391512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.392128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.426849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:50.443647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] MFA Rules not processed for user `13cc07e4481b47ad8add86a3e475a93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:50.447513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:50.674323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1115/2242] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:50.684716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:50.702152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:50.707076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:50.828336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cc47a-9152-45ac-a22e-bee88c2e6daf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:50.828825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4fb1c-e669-464c-b743-a77ff30070e2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:50.847348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.847348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.847348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.847348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.850337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.850337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.878600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:50.881216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:50.882612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:50.896011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] PATH_INFO: `/v3/projects/db25f0cca74b49ae954f1a057ce96d39` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:50.901979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db25f0cca74b49ae954f1a057ce96d39)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:50.912043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:50.917021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1128/2243] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:50 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:50.921316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:51.128460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f53c0-6b4d-482b-896a-702b47890004 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:51.143682 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db25f0cca74b49ae954f1a057ce96d39', 'request_id': 'req-28f83166-8924-4b80-adc0-da15fd4d488c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.166553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db25f0cca74b49ae954f1a057ce96d39', 'request_id': 'req-28f83166-8924-4b80-adc0-da15fd4d488c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.174077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db25f0cca74b49ae954f1a057ce96d39', 'request_id': 'req-28f83166-8924-4b80-adc0-da15fd4d488c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.194659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] The token cache is being invalidate because project db25f0cca74b49ae954f1a057ce96d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:51.195567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.197078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.205727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1116/2244] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:50 2022] DELETE /identity/v3/projects/db25f0cca74b49ae954f1a057ce96d39 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:51.206696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:51.208873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:51.247614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:51.248377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:51.248917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:51.249300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:51.250249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:51.254987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:51.291253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:51.293558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1129/2245] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:50 2022] POST /identity/v3/users => generated 292 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:51.297469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:51.300390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:51.316650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55cbd6-9f46-459e-9083-1173a7232672 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:51.322625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:51.323444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:51.323883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:51.324259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:51.325030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:51.326760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:51.343652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1130/2246] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:51.346304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:51.364133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f83166-8924-4b80-adc0-da15fd4d488c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:51.364878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:51.374036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:51.374994 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:51.375450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:51.375917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] PATH_INFO: `/v3/users/4bfa2fe10e57487fabe69776445cd1d5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:51.376047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa18b88-f596-4b43-92f6-a78b2b4a8df1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:51.377501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bfa2fe10e57487fabe69776445cd1d5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:51.380388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:51.381376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:51.382568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:51.382919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:51.383269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:51.384515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:51.386208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:51.402783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1131/2247] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:04:51.406889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:51.431453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f15a86-3acd-4f2f-9227-756d8b5baf34 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:51.456184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee448659-3fbe-4480-b487-0eadfb7ca1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:51.456882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee448659-3fbe-4480-b487-0eadfb7ca1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:51.457166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee448659-3fbe-4480-b487-0eadfb7ca1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:51.457446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee448659-3fbe-4480-b487-0eadfb7ca1c9 admin admin] PATH_INFO: `/v3/projects/20f5b3cf36174300bafa0dc406f24690/users/3f48a69de99841bcbe733a29591b2eb8/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:51.484887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee448659-3fbe-4480-b487-0eadfb7ca1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f5b3cf36174300bafa0dc406f24690, user_id=3f48a69de99841bcbe733a29591b2eb8, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:51.490172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee448659-3fbe-4480-b487-0eadfb7ca1c9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:51.495101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bfa2fe10e57487fabe69776445cd1d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.507946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bfa2fe10e57487fabe69776445cd1d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.508811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bfa2fe10e57487fabe69776445cd1d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.521232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bfa2fe10e57487fabe69776445cd1d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.531733 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1132/2248] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:51 2022] PUT /identity/v3/projects/20f5b3cf36174300bafa0dc406f24690/users/3f48a69de99841bcbe733a29591b2eb8/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:51.537419 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:51.537708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:51.540915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:51.573017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bfa2fe10e57487fabe69776445cd1d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:51.596038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1117/2249] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:51 2022] DELETE /identity/v3/users/4bfa2fe10e57487fabe69776445cd1d5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:51.626127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] MFA Rules not processed for user `3f48a69de99841bcbe733a29591b2eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:51.627468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:51.825939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1133/2250] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:51.830437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:51.833675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:51.985384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c766e-84fc-4151-9e7f-cb95742e6351 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.000563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.001197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.001487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.001754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.002393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.004610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.084915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:52.093400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1118/2251] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:04:51 2022] POST /identity/v3/projects => generated 383 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:52.100052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ef35-058c-499b-b446-b717c4633c54 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.260176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.262204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.263328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.264679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.267220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.275214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.335667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:52.343188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1134/2252] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:04:52 2022] POST /identity/v3/users => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:04:52.353926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac6217-c510-40b3-89a5-133d9cfd3a2f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.369314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.373700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.374277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.374863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.375736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.378226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.397272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:52.408662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1119/2253] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:52.410834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d5324-9c06-4730-b2f0-7a55d9ba8647 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.417216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.422832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.423356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.423803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] PATH_INFO: `/v3/projects/b6a8f4f6615e4f96b1326bd5afb39537/users/09cb41938fbd4a35966dc65fa1768074/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.456785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a8f4f6615e4f96b1326bd5afb39537, user_id=09cb41938fbd4a35966dc65fa1768074, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.461653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.479491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:52.485416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:52.510206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f0a9-778a-4c32-8f88-424db1595a8c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.510963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1135/2254] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:52 2022] PUT /identity/v3/projects/b6a8f4f6615e4f96b1326bd5afb39537/users/09cb41938fbd4a35966dc65fa1768074/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:52.514720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:52.561770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:52.606961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:52.625331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab80ac-8824-4a05-8eb7-425f63a30977 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.633023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.633884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.634369 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.634809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.635671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.637766 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.652571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1136/2255] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:04:52.660903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:52.703954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4731-9ba2-40da-a833-c99afe55fd26 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:52.719836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d825-074a-41c7-a568-bc39a85d20c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.721853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d825-074a-41c7-a568-bc39a85d20c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.723171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d825-074a-41c7-a568-bc39a85d20c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.723584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d825-074a-41c7-a568-bc39a85d20c0 admin admin] PATH_INFO: `/v3/projects/b6a8f4f6615e4f96b1326bd5afb39537/users/09cb41938fbd4a35966dc65fa1768074/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.732691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0d825-074a-41c7-a568-bc39a85d20c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a8f4f6615e4f96b1326bd5afb39537, user_id=09cb41938fbd4a35966dc65fa1768074, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.745147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:52.745942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.746444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.746727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] PATH_INFO: `/v3/projects/6952bed394c543d79031bce6a0b16884` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.748677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6952bed394c543d79031bce6a0b16884)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:52.750709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.751110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0d825-074a-41c7-a568-bc39a85d20c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:52.792513 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1137/2256] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:04:52 2022] PUT /identity/v3/projects/b6a8f4f6615e4f96b1326bd5afb39537/users/09cb41938fbd4a35966dc65fa1768074/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:52.863694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6952bed394c543d79031bce6a0b16884', 'request_id': 'req-74b06e38-1efa-4676-9021-7439520482d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:52.872509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6952bed394c543d79031bce6a0b16884', 'request_id': 'req-74b06e38-1efa-4676-9021-7439520482d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:52.873987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:52.874275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:52.874552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:52.879522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6952bed394c543d79031bce6a0b16884', 'request_id': 'req-74b06e38-1efa-4676-9021-7439520482d5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:52.900407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] The token cache is being invalidate because project 6952bed394c543d79031bce6a0b16884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:52.900793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:52.901594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:52.903326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1120/2257] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:52 2022] DELETE /identity/v3/projects/6952bed394c543d79031bce6a0b16884 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:52.936646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] MFA Rules not processed for user `09cb41938fbd4a35966dc65fa1768074`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:52.937926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:53.043506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1138/2258] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:53.058886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:53.060618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:53.196487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b06e38-1efa-4676-9021-7439520482d5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:53.200738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:53.201732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:53.201732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:53.202217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:53.205080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:53.356284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:53.357793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1a6bd-919c-4d53-8be7-2fd5825263e9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:53.369879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1121/2259] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:53.866609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:53.868252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:53.966609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fe400-5c58-4b6f-a8e9-e9ccf6ade91e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:53.973606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:53.974669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:53.975095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:53.976176 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] PATH_INFO: `/v3/users/e90017290ec545a088568cd802c62d75` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:53.977548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90017290ec545a088568cd802c62d75)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:53.980572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:54.028662 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:54.044699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90017290ec545a088568cd802c62d75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:54.055240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90017290ec545a088568cd802c62d75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:54.068016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90017290ec545a088568cd802c62d75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:54.069078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90017290ec545a088568cd802c62d75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:54.085604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90017290ec545a088568cd802c62d75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:54.097264 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1139/2260] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:04:53 2022] DELETE /identity/v3/users/e90017290ec545a088568cd802c62d75 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:54.099221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:54.101306 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:54.101776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:54.132899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:54.134314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:54.154719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:54.228879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1122/2261] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:04:54.238761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:54.240286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:54.309746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc06d9-181d-4781-b7eb-9bdd26a813af admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:54.313711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:54.314514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:54.314804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:54.315084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:54.329708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:54.331097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:54.343864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1140/2262] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:54.920762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:54.922702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:54.942472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0269c-9cb3-40dd-b73d-4684ffeaa890 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:54.960203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:54.961006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:54.961418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:54.962271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] PATH_INFO: `/v3/projects/a26ace4e3d0747f6b5dc8687003a2e34` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:54.963297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26ace4e3d0747f6b5dc8687003a2e34)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:54.965459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:55.068167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26ace4e3d0747f6b5dc8687003a2e34', 'request_id': 'req-e0a4d9d4-a786-4634-97fb-f0a73c177232'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:55.081460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26ace4e3d0747f6b5dc8687003a2e34', 'request_id': 'req-e0a4d9d4-a786-4634-97fb-f0a73c177232'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:55.093911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26ace4e3d0747f6b5dc8687003a2e34', 'request_id': 'req-e0a4d9d4-a786-4634-97fb-f0a73c177232'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:55.126258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] The token cache is being invalidate because project a26ace4e3d0747f6b5dc8687003a2e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:04:55.127371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:55.129361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:04:55.134072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1123/2263] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:04:54 2022] DELETE /identity/v3/projects/a26ace4e3d0747f6b5dc8687003a2e34 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:04:55.174592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:55.175213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:55.444886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d56e88-ff25-4173-a91f-2a60dce8ce5d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:55.645753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:55.646383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:55.646667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:55.646942 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:55.651029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:55.709184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:55.764346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:55.766557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba491c79-73b7-4d21-ac90-7110e5224c10 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:55.793698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1141/2264] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:56.762171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:56.763647 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:56.788860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4d9d4-a786-4634-97fb-f0a73c177232 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:56.804042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:56.805430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:56.805872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:56.806299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:56.809028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:56.914063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:56.916295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:56.927588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1124/2265] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:04:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:04:58.638744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:58.639863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:58.640354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:58.676946 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:04:58.678267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:58.791639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1142/2266] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:04:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:04:58.821010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:04:58.822567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:04:58.866129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:04:58.922460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0a297-14bb-4b7c-9e86-8471c99800db service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:04:58.928987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93905048-3ef6-4fbd-8b19-31629745361b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:04:58.929855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93905048-3ef6-4fbd-8b19-31629745361b service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:04:58.930303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93905048-3ef6-4fbd-8b19-31629745361b service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:04:58.930761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93905048-3ef6-4fbd-8b19-31629745361b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:04:58.946510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93905048-3ef6-4fbd-8b19-31629745361b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:04:58.948239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93905048-3ef6-4fbd-8b19-31629745361b service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:04:58.967805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1125/2267] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:04:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:05:01.689500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:01.691073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:01.833895 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb60bb-f854-4560-83b5-c569b68909cc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:01.845698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:01.846413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:01.846719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:01.846970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] PATH_INFO: `/v3/users/13cc07e4481b47ad8add86a3e475a93c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:01.851749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13cc07e4481b47ad8add86a3e475a93c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:01.860551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:01.960163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13cc07e4481b47ad8add86a3e475a93c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:01.979643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13cc07e4481b47ad8add86a3e475a93c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:01.997716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13cc07e4481b47ad8add86a3e475a93c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:02.000020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13cc07e4481b47ad8add86a3e475a93c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:02.028148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:02.029628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13cc07e4481b47ad8add86a3e475a93c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:02.042319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1143/2268] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:05:01 2022] DELETE /identity/v3/users/13cc07e4481b47ad8add86a3e475a93c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:02.046477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:02.047429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:02.047948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:02.088478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:02.090113 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:02.204568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1126/2269] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:02.205363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:02.206789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:02.352172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7a03-061e-40fc-91ef-e471334be7b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:02.364085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:02.365193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:02.366085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:02.366439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:02.392215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:02.396911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:02.427570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1144/2270] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:03.585181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:03.586709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:03.611446 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c1148-a5ff-4c75-b631-db3bf160c013 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:03.617080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:03.617732 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:03.618009 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:03.618283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] PATH_INFO: `/v3/projects/c5534652a6ac48d6b371756c9197d207` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:03.620706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5534652a6ac48d6b371756c9197d207)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:03.622695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:03.742344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5534652a6ac48d6b371756c9197d207', 'request_id': 'req-93505a2c-60f0-4d9f-a448-94eff9aef2e9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:03.757589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5534652a6ac48d6b371756c9197d207', 'request_id': 'req-93505a2c-60f0-4d9f-a448-94eff9aef2e9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:03.765193 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5534652a6ac48d6b371756c9197d207', 'request_id': 'req-93505a2c-60f0-4d9f-a448-94eff9aef2e9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:03.786822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] The token cache is being invalidate because project c5534652a6ac48d6b371756c9197d207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:05:03.787124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:03.788474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:03.791936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1127/2271] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:05:03 2022] DELETE /identity/v3/projects/c5534652a6ac48d6b371756c9197d207 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:03.796107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:03.842294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88c3d-44b6-4d8e-92a4-33a52e86063f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:03.996981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:04.001197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:04.001736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:04.002202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] PATH_INFO: `/v3/users/3f48a69de99841bcbe733a29591b2eb8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:04.003695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f48a69de99841bcbe733a29591b2eb8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:04.008859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:04.117193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f48a69de99841bcbe733a29591b2eb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:04.146510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f48a69de99841bcbe733a29591b2eb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:04.182683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f48a69de99841bcbe733a29591b2eb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:04.187136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f48a69de99841bcbe733a29591b2eb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:04.240716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f48a69de99841bcbe733a29591b2eb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:04.274652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1145/2272] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:05:03 2022] DELETE /identity/v3/users/3f48a69de99841bcbe733a29591b2eb8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:04.921107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:04.923892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:04.948664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93505a2c-60f0-4d9f-a448-94eff9aef2e9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:05.000511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:05.027528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:05.028911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:05.029360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:05.029792 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] PATH_INFO: `/v3/projects/20f5b3cf36174300bafa0dc406f24690` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:05.031957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f5b3cf36174300bafa0dc406f24690)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:05.034600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:05.132377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f5b3cf36174300bafa0dc406f24690', 'request_id': 'req-e389adfe-b4e3-44dc-b926-01ad07ca45c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.141947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f5b3cf36174300bafa0dc406f24690', 'request_id': 'req-e389adfe-b4e3-44dc-b926-01ad07ca45c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.153290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f5b3cf36174300bafa0dc406f24690', 'request_id': 'req-e389adfe-b4e3-44dc-b926-01ad07ca45c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.174037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] The token cache is being invalidate because project 20f5b3cf36174300bafa0dc406f24690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:05:05.174461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.175767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.178470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1128/2273] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:05:04 2022] DELETE /identity/v3/projects/20f5b3cf36174300bafa0dc406f24690 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:05.182338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:05.183941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:05.201555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798d1af-f1e3-4175-b955-4d10f107b55d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:05.221850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:05.296544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:05.297951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:05.298404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:05.298845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] PATH_INFO: `/v3/users/09cb41938fbd4a35966dc65fa1768074` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:05.300263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] RBAC: Authorizing `identity:delete_user(user_id=09cb41938fbd4a35966dc65fa1768074)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:05.302651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:05.365880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09cb41938fbd4a35966dc65fa1768074'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.379501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09cb41938fbd4a35966dc65fa1768074'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.388970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09cb41938fbd4a35966dc65fa1768074'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.389732 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09cb41938fbd4a35966dc65fa1768074'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.408641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-657edebe-ce39-4683-bcd0-6a2ffccb68da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09cb41938fbd4a35966dc65fa1768074'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:05.423344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1146/2274] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:05:05 2022] DELETE /identity/v3/users/09cb41938fbd4a35966dc65fa1768074 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:05:05.951986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:05.953455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:05.977042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389adfe-b4e3-44dc-b926-01ad07ca45c9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:06.053723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:06.054484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:06.054764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:06.057534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] PATH_INFO: `/v3/projects/b6a8f4f6615e4f96b1326bd5afb39537` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:06.061649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a8f4f6615e4f96b1326bd5afb39537)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:06.065404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:06.140947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a8f4f6615e4f96b1326bd5afb39537', 'request_id': 'req-6f45b9e4-bd41-4617-9d78-62af043358fd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:06.151583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a8f4f6615e4f96b1326bd5afb39537', 'request_id': 'req-6f45b9e4-bd41-4617-9d78-62af043358fd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:06.161799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a8f4f6615e4f96b1326bd5afb39537', 'request_id': 'req-6f45b9e4-bd41-4617-9d78-62af043358fd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:06.180923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:06.191572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] The token cache is being invalidate because project b6a8f4f6615e4f96b1326bd5afb39537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:05:06.192296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:06.194246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:06.212561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1129/2275] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:05:05 2022] DELETE /identity/v3/projects/b6a8f4f6615e4f96b1326bd5afb39537 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:06.405911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec36ecd-1ec5-4c5d-8d86-355a6556388c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:06.406428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec36ecd-1ec5-4c5d-8d86-355a6556388c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:06.406508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec36ecd-1ec5-4c5d-8d86-355a6556388c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:06.452032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ec36ecd-1ec5-4c5d-8d86-355a6556388c None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:06.453524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ec36ecd-1ec5-4c5d-8d86-355a6556388c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:06.621699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1147/2276] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 9 07:05:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:05:06.650187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:06.654838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:06.797970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45b9e4-bd41-4617-9d78-62af043358fd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:06.809297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddb528-ff07-4349-b49e-a576e8296a9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:06.811960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb528-ff07-4349-b49e-a576e8296a9d service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:06.811960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb528-ff07-4349-b49e-a576e8296a9d service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:06.812213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb528-ff07-4349-b49e-a576e8296a9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:06.825194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb528-ff07-4349-b49e-a576e8296a9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:06.826901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb528-ff07-4349-b49e-a576e8296a9d service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:06.837140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1130/2277] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:05:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:05:09.594072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a655b5-46a8-419c-bd27-171821151655 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:09.594741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a655b5-46a8-419c-bd27-171821151655 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:09.594741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a655b5-46a8-419c-bd27-171821151655 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:09.596365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1148/2278] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:05:09.634356 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:09.634897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:09.634897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:09.661743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] MFA Rules not processed for user `86a917f8f79a494bbd8c18fbefe695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:09.668911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:09.714412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a655b5-46a8-419c-bd27-171821151655 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:09.716499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73a655b5-46a8-419c-bd27-171821151655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:09.770336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:09.803056 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1131/2279] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:05:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:09.810825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a655b5-46a8-419c-bd27-171821151655 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:09.816297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:09.816959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:09.817117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:09.817231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:09.817506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] PATH_INFO: `/v3/users/2a6b55d7cb1840a5ad93369589a36596` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:09.819262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6b55d7cb1840a5ad93369589a36596)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:09.820427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:09.821992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:09.841479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57339c2-7d00-4862-aa21-5e752e6ac25d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:09.850890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:09.851936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:09.852417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:09.852740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:09.868246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:09.870124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:09.888572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6b55d7cb1840a5ad93369589a36596'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:09.892918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1132/2280] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:05:09.898482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6b55d7cb1840a5ad93369589a36596'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:09.919311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6b55d7cb1840a5ad93369589a36596'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:09.920177 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6b55d7cb1840a5ad93369589a36596'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:09.937523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74ddcbd4-75d9-42dc-8acf-d00dfe09117b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6b55d7cb1840a5ad93369589a36596'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:09.948217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1149/2281] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:05:09 2022] DELETE /identity/v3/users/2a6b55d7cb1840a5ad93369589a36596 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:05:10.205918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.228914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65461c-63f8-44a2-8d2b-43a8d88393aa service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.321760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.322886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.323404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.324022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] PATH_INFO: `/v3/projects/814c528dd56c4e61ae5cba5733bf2f08` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.325522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=814c528dd56c4e61ae5cba5733bf2f08)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.328961 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.349693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:10.405771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814c528dd56c4e61ae5cba5733bf2f08', 'request_id': 'req-1cb77b8a-0890-46c3-aeba-f8c462729750'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:10.417144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814c528dd56c4e61ae5cba5733bf2f08', 'request_id': 'req-1cb77b8a-0890-46c3-aeba-f8c462729750'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:10.424063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814c528dd56c4e61ae5cba5733bf2f08', 'request_id': 'req-1cb77b8a-0890-46c3-aeba-f8c462729750'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:10.440151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] The token cache is being invalidate because project 814c528dd56c4e61ae5cba5733bf2f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:05:10.440536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:10.441101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb77b8a-0890-46c3-aeba-f8c462729750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:10.446999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1133/2282] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:05:10 2022] DELETE /identity/v3/projects/814c528dd56c4e61ae5cba5733bf2f08 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:10.448607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.448994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.449343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.475607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:10.476880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:10.555106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1150/2283] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:05:10.557098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.557390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.557680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.581305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:10.582370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:10.592187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1134/2284] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:10.596455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.607236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4bb43-cb7b-433b-b2a2-8c4594e73a8a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.613691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.614414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.614829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.615820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.616582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:05:10.617000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.618694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.631628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1151/2285] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:05:10.636520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.646094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281664e0-ad9d-4d98-ad11-682934e2dfd1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.652087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.652723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.653024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.653231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.653867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.655895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.688278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1135/2286] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:05:10 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:05:10.692141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.704212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2439b5e-1656-4e4b-9610-fb1da3d0dec2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.759049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:10.780482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.781107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.781663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.781947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.782595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.784594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.812175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1152/2287] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:05:10 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:05:10.816424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.828334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34db93f-1560-4646-890c-29b7039ab59f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.833765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.834490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.834854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.835135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.835841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.837441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.851633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1136/2288] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:05:10.853286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.866832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789bd42-ee83-44b9-968c-e5a5cc94bf42 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.872638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c48fc-fc3e-46fa-b8db-5b5934ce9b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.873348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c48fc-fc3e-46fa-b8db-5b5934ce9b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.873742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c48fc-fc3e-46fa-b8db-5b5934ce9b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.874086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c48fc-fc3e-46fa-b8db-5b5934ce9b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.874864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c48fc-fc3e-46fa-b8db-5b5934ce9b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.876675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c48fc-fc3e-46fa-b8db-5b5934ce9b87 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.890202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1153/2289] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:05:10.894161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:10.905205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccfd7e-a026-4601-a14d-97781026d734 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:10.910032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:10.910693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.910981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.911249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] PATH_INFO: `/v3/projects/99bb0ffb36864603880d4e461620f572/users/4ab83a38a43346ad85c560bf130f43ff/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.926596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bb0ffb36864603880d4e461620f572, user_id=4ab83a38a43346ad85c560bf130f43ff, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:10.929845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:10.952097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1137/2290] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:05:10 2022] PUT /identity/v3/projects/99bb0ffb36864603880d4e461620f572/users/4ab83a38a43346ad85c560bf130f43ff/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:05:10.962788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:10.963108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:10.963363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:10.995586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] MFA Rules not processed for user `4ab83a38a43346ad85c560bf130f43ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:10.996864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:11.078490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1154/2291] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:05:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:11.090186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:11.093667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:11.163135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b122e85-e68f-4d96-ac6b-bca7454d2162 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:11.167719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec5314-5530-4f03-bb15-bb5a2e26b6a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:11.168341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec5314-5530-4f03-bb15-bb5a2e26b6a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:11.168621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec5314-5530-4f03-bb15-bb5a2e26b6a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:11.168904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec5314-5530-4f03-bb15-bb5a2e26b6a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:11.179400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec5314-5530-4f03-bb15-bb5a2e26b6a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:11.180872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec5314-5530-4f03-bb15-bb5a2e26b6a2 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:11.190750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1138/2292] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:11.218232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:11.293448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4984-bca7-47e4-b8ba-f4b6bd126539 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:11.298879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:11.299562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:11.299954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:11.300171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] PATH_INFO: `/v3/users/4ab83a38a43346ad85c560bf130f43ff` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:11.301464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab83a38a43346ad85c560bf130f43ff)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:11.303571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:11.359257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab83a38a43346ad85c560bf130f43ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:11.363299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:11.369774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab83a38a43346ad85c560bf130f43ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:11.380418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab83a38a43346ad85c560bf130f43ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:11.382301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab83a38a43346ad85c560bf130f43ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:11.416528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab83a38a43346ad85c560bf130f43ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:11.427658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1155/2293] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:05:11 2022] DELETE /identity/v3/users/4ab83a38a43346ad85c560bf130f43ff => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:11.432360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:11.432785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:11.433168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:11.483655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:11.483655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:11.543465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:11.580074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1139/2294] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:11.590864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:11.592397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:11.665049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdcb1-e142-456e-a696-2996a4f9064c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:11.668790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ea6b-d489-4b4d-a5e5-2a09ba2328aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:11.669884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ea6b-d489-4b4d-a5e5-2a09ba2328aa service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:11.669884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ea6b-d489-4b4d-a5e5-2a09ba2328aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:11.670088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ea6b-d489-4b4d-a5e5-2a09ba2328aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:11.680214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12ea6b-d489-4b4d-a5e5-2a09ba2328aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:11.681677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12ea6b-d489-4b4d-a5e5-2a09ba2328aa service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:11.695842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1156/2295] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:12.007946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:12.009408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:12.025451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d5cd9-0b3b-429f-909f-17b94f2c5acc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:12.034697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:12.035360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:12.036794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:12.037766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] PATH_INFO: `/v3/projects/99bb0ffb36864603880d4e461620f572` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:12.039078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bb0ffb36864603880d4e461620f572)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:12.040986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:12.097885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bb0ffb36864603880d4e461620f572', 'request_id': 'req-d3891651-d8e6-4886-b692-18ebb6dd7084'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:12.104209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bb0ffb36864603880d4e461620f572', 'request_id': 'req-d3891651-d8e6-4886-b692-18ebb6dd7084'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:12.110685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bb0ffb36864603880d4e461620f572', 'request_id': 'req-d3891651-d8e6-4886-b692-18ebb6dd7084'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:12.125772 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] The token cache is being invalidate because project 99bb0ffb36864603880d4e461620f572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:05:12.126074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:12.126726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d3891651-d8e6-4886-b692-18ebb6dd7084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:05:12.128472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1140/2296] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:05:12 2022] DELETE /identity/v3/projects/99bb0ffb36864603880d4e461620f572 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:05:17.016342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41a01-20ef-4ce9-a09c-40d5b31aac94 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.016932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41a01-20ef-4ce9-a09c-40d5b31aac94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.016932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41a01-20ef-4ce9-a09c-40d5b31aac94 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.018224 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1157/2297] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:05:17.023958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09e7e9-ec5e-4459-a0c7-2cd0708b6497 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.024412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09e7e9-ec5e-4459-a0c7-2cd0708b6497 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.024494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09e7e9-ec5e-4459-a0c7-2cd0708b6497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.046676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3a09e7e9-ec5e-4459-a0c7-2cd0708b6497 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:17.047932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a09e7e9-ec5e-4459-a0c7-2cd0708b6497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:17.117593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1141/2298] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:05:17.129593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea41a01-20ef-4ce9-a09c-40d5b31aac94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:17.131267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cea41a01-20ef-4ce9-a09c-40d5b31aac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:17.188887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea41a01-20ef-4ce9-a09c-40d5b31aac94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:17.192335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf66534-ab25-45ad-8924-e391fd20208b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:17.192936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf66534-ab25-45ad-8924-e391fd20208b service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.193215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf66534-ab25-45ad-8924-e391fd20208b service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.193485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf66534-ab25-45ad-8924-e391fd20208b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.203541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf66534-ab25-45ad-8924-e391fd20208b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:17.205086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf66534-ab25-45ad-8924-e391fd20208b service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:17.215853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1158/2299] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:17.283930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f7b49-dd11-4292-bb8b-531fcca5bbf4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.284213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f7b49-dd11-4292-bb8b-531fcca5bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.284485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f7b49-dd11-4292-bb8b-531fcca5bbf4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.285847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1142/2300] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:05:17.290982 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c998a-2317-4850-872a-b2f89f8ddb51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.291304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c998a-2317-4850-872a-b2f89f8ddb51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.291742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c998a-2317-4850-872a-b2f89f8ddb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.311891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1f7c998a-2317-4850-872a-b2f89f8ddb51 None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:17.313105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f7c998a-2317-4850-872a-b2f89f8ddb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:17.363401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1159/2301] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:05:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:17.375332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f7b49-dd11-4292-bb8b-531fcca5bbf4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:17.377200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ee2f7b49-dd11-4292-bb8b-531fcca5bbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:17.390681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f7b49-dd11-4292-bb8b-531fcca5bbf4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:17.394412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013c3d6-2c85-4408-bc04-c9244cd297ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:17.395005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c3d6-2c85-4408-bc04-c9244cd297ab service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.395284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c3d6-2c85-4408-bc04-c9244cd297ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.395553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c3d6-2c85-4408-bc04-c9244cd297ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.403867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c3d6-2c85-4408-bc04-c9244cd297ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:17.405274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c3d6-2c85-4408-bc04-c9244cd297ab service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:17.412988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1143/2302] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:17.768285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b80e01-07f3-471f-a54d-a7121a9171d3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.768573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b80e01-07f3-471f-a54d-a7121a9171d3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.768852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b80e01-07f3-471f-a54d-a7121a9171d3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.770219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1160/2303] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:05:17.776333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2564e-4c84-40fd-a848-d6261facdb2b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.776612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2564e-4c84-40fd-a848-d6261facdb2b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.776881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2564e-4c84-40fd-a848-d6261facdb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.795837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8db2564e-4c84-40fd-a848-d6261facdb2b None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:17.797017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8db2564e-4c84-40fd-a848-d6261facdb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:17.806265 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1144/2304] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:17.816502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b80e01-07f3-471f-a54d-a7121a9171d3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:17.827921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b80e01-07f3-471f-a54d-a7121a9171d3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:17.831452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:17.832083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:17.832361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:17.832645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:17.844588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:17.846012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:17.856697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1161/2305] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:20.207510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4947-ecd0-41d9-8b0c-9a8ca4c69c9c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:20.208014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4947-ecd0-41d9-8b0c-9a8ca4c69c9c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:20.208130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4947-ecd0-41d9-8b0c-9a8ca4c69c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:20.227948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-74ed4947-ecd0-41d9-8b0c-9a8ca4c69c9c None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:20.229154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-74ed4947-ecd0-41d9-8b0c-9a8ca4c69c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:20.261355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:20.286781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1145/2306] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:05:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:20.298907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:20.300310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:20.314148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:20.370233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f0dd-39f2-4aee-a290-eb467c0504af service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:20.375196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:20.375827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:20.376112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:20.376391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:20.385164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:20.386742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:20.394760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1162/2307] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:05:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:05:30.280034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:30.391467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=27 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:05:45.140425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:45.140831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:45.141031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:45.159747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:05:45.160949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:45.168280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1146/2308] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 9 07:05:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:05:45.190040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:45.201632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20808503-cfa0-4d36-830b-cfc4da7bae3f service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:45.206563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4825-a484-4367-9fef-c0f4dc47295d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:45.207147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4825-a484-4367-9fef-c0f4dc47295d service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:45.207423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4825-a484-4367-9fef-c0f4dc47295d service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:45.207737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4825-a484-4367-9fef-c0f4dc47295d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:45.215999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4825-a484-4367-9fef-c0f4dc47295d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:45.217590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4825-a484-4367-9fef-c0f4dc47295d service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:45.225985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1163/2309] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:05:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:05:53.647839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:05:53.650010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:53.736287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56455cd9-fbd8-4a10-b9f7-3ad1df18f6ad None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:05:53.743369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:05:53.744234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:05:53.744899 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:05:53.745327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:05:53.748007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:05:53.762345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:05:53.764533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254564e-4884-48f2-89fa-bf7cefcba504 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:05:53.776507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1147/2310] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:05:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:05:55.222262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:06:03.772192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=35 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:06:07.571993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8556a-9713-47bd-9208-4765ff064bce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:06:07.572515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8556a-9713-47bd-9208-4765ff064bce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:06:07.572644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8556a-9713-47bd-9208-4765ff064bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:06:07.592053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-33b8556a-9713-47bd-9208-4765ff064bce None None] MFA Rules not processed for user `cc885026afe041cca086367bb67d8a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:06:07.593274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33b8556a-9713-47bd-9208-4765ff064bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:06:07.655774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1164/2311] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:06:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:06:17.648863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:06:27.443122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d7a7-50c8-404e-a504-14e18cb8af97 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:06:27.443677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d7a7-50c8-404e-a504-14e18cb8af97 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:06:27.443779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d7a7-50c8-404e-a504-14e18cb8af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:06:27.464445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e644d7a7-50c8-404e-a504-14e18cb8af97 None None] MFA Rules not processed for user `19da3c426c6e4c4a889ab5017804c92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:06:27.465654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e644d7a7-50c8-404e-a504-14e18cb8af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:06:27.521496 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1148/2312] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:06:34.557813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc438229-dbf6-4ad8-be81-2dafdb43af06 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:06:34.558375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc438229-dbf6-4ad8-be81-2dafdb43af06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:06:34.558479 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc438229-dbf6-4ad8-be81-2dafdb43af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:06:34.576888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cc438229-dbf6-4ad8-be81-2dafdb43af06 None None] MFA Rules not processed for user `0bb72c5f52fc498aa60eecba3bd5248a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:06:34.578082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc438229-dbf6-4ad8-be81-2dafdb43af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:06:34.633033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1165/2313] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:06:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:06:37.515015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:06:44.626166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:01.841845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:01.842343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:01.842439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:01.862161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] MFA Rules not processed for user `4e528f53a4b94e0fab0d90e0dbe07216`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:01.863394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:01.917933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1149/2314] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:11.911358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:17.072508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:17.073017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:17.073165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:17.094112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] MFA Rules not processed for user `cc885026afe041cca086367bb67d8a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:17.095352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:17.102493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1166/2315] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:22.654962 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:22.664524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191774b4-4576-4394-9c15-0fa9c5a4cbee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:22.668107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:22.669506 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:22.669631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:22.669863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:22.671581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:22.685561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:22.686985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:22.697779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1150/2316] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:23.871021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:23.881528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791842-71f7-4d1b-9db1-59647ff2496b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:23.884997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:23.885585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:23.885867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:23.886135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:23.888547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:23.902450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:23.905117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:23.916333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1167/2317] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:32.693021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:33.012505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:33.014514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:33.025988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db357199-f4e4-46fc-a1db-910e1fe23777 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:33.030448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:33.031044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:33.031320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:33.031589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:33.033322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:33.115141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:33.116594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:33.127280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1151/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:33.911355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:36.399794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:36.401270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:36.416662 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aeb35-a4a5-4786-bc80-a2de77335b08 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:36.421478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:36.422107 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:36.422404 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:36.422665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] PATH_INFO: `/v3/users/19da3c426c6e4c4a889ab5017804c92e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:36.423835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19da3c426c6e4c4a889ab5017804c92e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:36.425735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:36.472053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19da3c426c6e4c4a889ab5017804c92e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.479138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19da3c426c6e4c4a889ab5017804c92e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.486260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19da3c426c6e4c4a889ab5017804c92e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.486892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19da3c426c6e4c4a889ab5017804c92e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.499616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f03f5b1-ceed-4ba8-b6db-391422ff786a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19da3c426c6e4c4a889ab5017804c92e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.505576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1168/2319] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:07:36 2022] DELETE /identity/v3/users/19da3c426c6e4c4a889ab5017804c92e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:36.718767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:36.729968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4a390-e2e7-4d98-9408-6159e954faeb service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:36.801859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:36.802491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:36.802774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:36.803041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] PATH_INFO: `/v3/projects/517f3dedc5da431b87e398003506c06b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:36.804176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517f3dedc5da431b87e398003506c06b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:36.806090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:36.852346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517f3dedc5da431b87e398003506c06b', 'request_id': 'req-284b9060-06d7-467f-aa55-66a50501cd54'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.859856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517f3dedc5da431b87e398003506c06b', 'request_id': 'req-284b9060-06d7-467f-aa55-66a50501cd54'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.866674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517f3dedc5da431b87e398003506c06b', 'request_id': 'req-284b9060-06d7-467f-aa55-66a50501cd54'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.879592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] The token cache is being invalidate because project 517f3dedc5da431b87e398003506c06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:07:36.879910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.880532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-284b9060-06d7-467f-aa55-66a50501cd54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:36.882074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1152/2320] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:07:36 2022] DELETE /identity/v3/projects/517f3dedc5da431b87e398003506c06b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:36.886952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:36.887245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:36.887526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:36.907653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:36.908861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:36.967548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1169/2321] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:36.975392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:36.975726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:36.975985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:36.995775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:36.996993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:37.005181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1153/2322] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:37.009385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.019591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97af71-3c77-47e8-8ea5-042222ce3a9a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.024172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.024767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.025049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.025321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.025964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:07:37.026266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.027905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.039892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1170/2323] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:07:37.044206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.053322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc56d1f-7d31-4eb7-9cf4-95a8e242edfc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.058028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.058621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.058902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.059174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.059877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.061876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.077907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:37.091525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1154/2324] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:07:37 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:37.095564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.105532 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d241-a92b-4cab-9fdd-73c960403bc0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.161273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.161878 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.162160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.162432 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.163066 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.165037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.173336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:37.188739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1171/2325] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:07:37 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:07:37.192836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.203290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8566d6-62e5-417e-85e3-a64090b0125a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.207942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.208569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.208857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.209132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.209773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.211346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.221088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1155/2326] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:37.224834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.235334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76567a-ba86-468f-982f-6e59b581eba4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.239941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11d082-d885-4acd-ab47-295e25442ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.240530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11d082-d885-4acd-ab47-295e25442ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.240810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11d082-d885-4acd-ab47-295e25442ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.241080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11d082-d885-4acd-ab47-295e25442ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.241713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11d082-d885-4acd-ab47-295e25442ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.243278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11d082-d885-4acd-ab47-295e25442ddb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.254420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1172/2327] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:07:37.258426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.268387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f722e3-b7ea-4cde-9354-796330738e6a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.273058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.273713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.273991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.274268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] PATH_INFO: `/v3/projects/f232c93b5d02445b90b67549e6bd4ce4/users/361528fac536441b953518ad4c02842e/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.289313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f232c93b5d02445b90b67549e6bd4ce4, user_id=361528fac536441b953518ad4c02842e, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.292038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.304191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1156/2328] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:37 2022] PUT /identity/v3/projects/f232c93b5d02445b90b67549e6bd4ce4/users/361528fac536441b953518ad4c02842e/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:37.314708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.315020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.315295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.344203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] MFA Rules not processed for user `361528fac536441b953518ad4c02842e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:37.345399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:37.409483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1173/2329] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:07:37.413996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.474054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb329222-eaa0-421c-8ad4-34c59c0f2616 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.478640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.479232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.479509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.479808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.480420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.482589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.508848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1157/2330] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:07:37 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:37.513554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.524445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838331d9-7abc-46d1-978f-7c8cffcfac48 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.581174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.582098 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.582385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.582667 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.583286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.585485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.606740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1174/2331] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:07:37 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:37.610850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.621283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9509b19-50ef-41b8-8a80-220181d6b2bb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.626620 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.627317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.627511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.628119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.628776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.630544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.641608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1158/2332] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:37.645812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.655963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681a01f-4696-4ab8-8d1e-f40a8b86952a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.660789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.661383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.661663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.661943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] PATH_INFO: `/v3/projects/6c86c78d02594d65af90ba602de0e6d7/users/d25f88f3f0a24cfc9a464eb3824425ae/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.677502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c86c78d02594d65af90ba602de0e6d7, user_id=d25f88f3f0a24cfc9a464eb3824425ae, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.680193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.693286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1175/2333] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:37 2022] PUT /identity/v3/projects/6c86c78d02594d65af90ba602de0e6d7/users/d25f88f3f0a24cfc9a464eb3824425ae/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:37.697358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.707498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7e225-4c9e-4f22-b095-dec03b557ed6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.763203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759d071-a0c0-40ac-a00a-690ce549c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.763998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759d071-a0c0-40ac-a00a-690ce549c6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.764098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759d071-a0c0-40ac-a00a-690ce549c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.764360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759d071-a0c0-40ac-a00a-690ce549c6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.765222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759d071-a0c0-40ac-a00a-690ce549c6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.766987 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759d071-a0c0-40ac-a00a-690ce549c6ea admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.776651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1159/2334] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:37.781549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.791961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3d282-b9bc-4953-8dd8-5bd660756d16 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:37.797140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:37.797797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.798080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.798356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] PATH_INFO: `/v3/projects/6c86c78d02594d65af90ba602de0e6d7/users/d25f88f3f0a24cfc9a464eb3824425ae/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.800288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c86c78d02594d65af90ba602de0e6d7, user_id=d25f88f3f0a24cfc9a464eb3824425ae, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:37.802997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:37.814357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1176/2335] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:37 2022] PUT /identity/v3/projects/6c86c78d02594d65af90ba602de0e6d7/users/d25f88f3f0a24cfc9a464eb3824425ae/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:37.825661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:37.825661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:37.826365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:37.855895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] MFA Rules not processed for user `d25f88f3f0a24cfc9a464eb3824425ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:37.858352 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:37.879747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:37.931118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1160/2336] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:37.943449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:37.945236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:37.969009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:38.009008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae58a9-0c91-4a5d-912a-ba1ac24511b1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:38.012672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666e70-211f-4039-b3e2-55501c4b2bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:38.013368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666e70-211f-4039-b3e2-55501c4b2bab service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:38.013791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666e70-211f-4039-b3e2-55501c4b2bab service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:38.014064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666e70-211f-4039-b3e2-55501c4b2bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:38.023170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39666e70-211f-4039-b3e2-55501c4b2bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:38.024622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39666e70-211f-4039-b3e2-55501c4b2bab service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:38.034670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1177/2337] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:38.414510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:38.478236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b40aad-e4ec-4363-bc4a-883fe2260dab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:38.483664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:38.484440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:38.484720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:38.485002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] PATH_INFO: `/v3/users/361528fac536441b953518ad4c02842e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:38.486122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361528fac536441b953518ad4c02842e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:38.488051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:38.535477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361528fac536441b953518ad4c02842e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:38.542709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361528fac536441b953518ad4c02842e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:38.543403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361528fac536441b953518ad4c02842e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:38.551085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361528fac536441b953518ad4c02842e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:38.563066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361528fac536441b953518ad4c02842e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:38.568870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1161/2338] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:07:38 2022] DELETE /identity/v3/users/361528fac536441b953518ad4c02842e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:38.573341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:38.573822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:38.573985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:38.594566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:38.595777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:38.654075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1178/2339] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:38.663780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:38.665204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:38.706599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:38.725685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f565a-3d4e-4fbe-a2f2-08175d778c92 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:38.729322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:38.729863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:38.730134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:38.730403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:38.739408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:38.740861 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:38.752392 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1162/2340] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:39.022394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:39.024425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:39.036770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f9e36-a60e-4b91-bb9e-62c053386812 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:39.043266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:39.043948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.044260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.045252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] PATH_INFO: `/v3/projects/f232c93b5d02445b90b67549e6bd4ce4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.046807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f232c93b5d02445b90b67549e6bd4ce4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:39.048944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:39.104118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f232c93b5d02445b90b67549e6bd4ce4', 'request_id': 'req-1c60264e-f2bf-4556-8f23-54b5876129f4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.113936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f232c93b5d02445b90b67549e6bd4ce4', 'request_id': 'req-1c60264e-f2bf-4556-8f23-54b5876129f4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.120993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f232c93b5d02445b90b67549e6bd4ce4', 'request_id': 'req-1c60264e-f2bf-4556-8f23-54b5876129f4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.132823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] The token cache is being invalidate because project f232c93b5d02445b90b67549e6bd4ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:07:39.133115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.133777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.135501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1179/2341] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:07:39 2022] DELETE /identity/v3/projects/f232c93b5d02445b90b67549e6bd4ce4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:39.140024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:39.149752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba8d99-8dfc-4562-8887-0273733e3f90 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:39.207133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:39.207800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.208082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.208358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] PATH_INFO: `/v3/users/d25f88f3f0a24cfc9a464eb3824425ae` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.209479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25f88f3f0a24cfc9a464eb3824425ae)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:39.211420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:39.257467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25f88f3f0a24cfc9a464eb3824425ae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.264697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25f88f3f0a24cfc9a464eb3824425ae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.265368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25f88f3f0a24cfc9a464eb3824425ae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.273128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25f88f3f0a24cfc9a464eb3824425ae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.284915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5d3fb128-33e5-4033-8b22-4f023721a380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25f88f3f0a24cfc9a464eb3824425ae'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.290715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1163/2342] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:07:39 2022] DELETE /identity/v3/users/d25f88f3f0a24cfc9a464eb3824425ae => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:07:39.556219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:39.557704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:39.572285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60264e-f2bf-4556-8f23-54b5876129f4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:39.617139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:39.632117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:39.632792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.633077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.633356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] PATH_INFO: `/v3/projects/6c86c78d02594d65af90ba602de0e6d7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.634503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c86c78d02594d65af90ba602de0e6d7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:39.636475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:39.682366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c86c78d02594d65af90ba602de0e6d7', 'request_id': 'req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.689034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c86c78d02594d65af90ba602de0e6d7', 'request_id': 'req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.695368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c86c78d02594d65af90ba602de0e6d7', 'request_id': 'req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.705756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] The token cache is being invalidate because project 6c86c78d02594d65af90ba602de0e6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:07:39.706047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.706687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-62f9cba8-2ac3-4b5a-9f2a-6d927e0c5a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:39.708196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1180/2343] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:07:39 2022] DELETE /identity/v3/projects/6c86c78d02594d65af90ba602de0e6d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:39.713059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.713345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.713627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.734164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:39.735382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:39.791205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:39.807705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1164/2344] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:39.810719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.811059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.811368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.830771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:39.831998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:39.839905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1181/2345] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:39.844785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:39.854814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e431a3-4e0d-458e-ab90-83ebf7ade271 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:39.860185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:39.860789 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.861065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.861771 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.862120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:07:39.862420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:39.864037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:39.875938 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1165/2346] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:07:39.880497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:39.890507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a2a58-9bd5-4602-85fc-1aab075010e2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:39.895212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:39.895850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.896201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.896477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.897113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:39.899005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:39.926231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1182/2347] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:07:39 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:39.930675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:39.940785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8f940-f927-48d9-b050-314ce6f6e8d4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:39.996444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:39.997039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:39.997365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:39.997663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:39.998291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.000232 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.021998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1166/2348] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:07:39 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:40.026146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.037154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481196-8307-494b-ba2f-c0b4143e373c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.041758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.042362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.042640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.042913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.043569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.045458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.058062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1183/2349] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:40.059868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.069703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697dbc-c1c5-4ea6-a4fe-19332a4ea55d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.074315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29ed0e-8f55-464e-bc52-1d38af560f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.074906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29ed0e-8f55-464e-bc52-1d38af560f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.075186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29ed0e-8f55-464e-bc52-1d38af560f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.076221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29ed0e-8f55-464e-bc52-1d38af560f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.076818 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29ed0e-8f55-464e-bc52-1d38af560f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.078397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29ed0e-8f55-464e-bc52-1d38af560f63 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.088952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1167/2350] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:40.093662 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.103435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0917e-7930-4c36-8b38-8263b5376fa0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.108209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.108851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.109125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.109398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] PATH_INFO: `/v3/projects/2650c6696ab943de93e11a6489c25987/users/d824954f4a3c45398380c716ea7e31f9/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.124085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2650c6696ab943de93e11a6489c25987, user_id=d824954f4a3c45398380c716ea7e31f9, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.127340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.140991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1184/2351] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:40 2022] PUT /identity/v3/projects/2650c6696ab943de93e11a6489c25987/users/d824954f4a3c45398380c716ea7e31f9/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:40.151130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.151638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.151749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.180532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] MFA Rules not processed for user `d824954f4a3c45398380c716ea7e31f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:40.181731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:40.251192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1168/2352] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:40.256140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.321047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265649bd-408a-4004-9052-782f2cfa26db admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.326775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.327383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.327687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.327963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.328583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.330480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.350872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:40.358976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1185/2353] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:07:40 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:40.363213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.375118 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98403b6e-1d87-45e4-b033-4f7d6f919cfa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.422816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:40.432809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.433435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.433711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.433984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.435128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.437271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.458826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1169/2354] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:07:40 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:40.462781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.473572 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b4b8-da23-4fad-86ca-fda6c05781d7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.478071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.478663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.478938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.479205 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.479865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.481465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.492382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1186/2355] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:40.496188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.506766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc402-8b38-4b7a-aa44-e6c71426195b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.511334 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.512007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.512289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.512561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] PATH_INFO: `/v3/projects/908857250f804089bee0ed0d0b9840eb/users/18280a24905e432d83a1645c99056754/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.527444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908857250f804089bee0ed0d0b9840eb, user_id=18280a24905e432d83a1645c99056754, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.530315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.542830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1170/2356] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:40 2022] PUT /identity/v3/projects/908857250f804089bee0ed0d0b9840eb/users/18280a24905e432d83a1645c99056754/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:07:40.546775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.557255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44b47a-437b-4101-aacc-c378c18ed92e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.612489 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b454e0f-33f0-46cb-bb7d-c1defc700f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.613023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b454e0f-33f0-46cb-bb7d-c1defc700f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.613305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b454e0f-33f0-46cb-bb7d-c1defc700f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.613580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b454e0f-33f0-46cb-bb7d-c1defc700f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.614228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b454e0f-33f0-46cb-bb7d-c1defc700f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.615855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b454e0f-33f0-46cb-bb7d-c1defc700f13 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.626018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1187/2357] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:40.630811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.641609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddea0ce-0721-420b-9aab-d344502c3e52 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.646188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.646836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.647117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.647388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] PATH_INFO: `/v3/projects/908857250f804089bee0ed0d0b9840eb/users/18280a24905e432d83a1645c99056754/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.649326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=908857250f804089bee0ed0d0b9840eb, user_id=18280a24905e432d83a1645c99056754, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.652034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.662068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1171/2358] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:40 2022] PUT /identity/v3/projects/908857250f804089bee0ed0d0b9840eb/users/18280a24905e432d83a1645c99056754/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:40.673967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.674280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.674552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.702930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] MFA Rules not processed for user `18280a24905e432d83a1645c99056754`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:40.704202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:40.771558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1188/2359] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:40.782434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:40.784281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:40.843460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f43be-25b5-424c-9e8c-c745e2c23aae admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:40.847099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b3c42-bc0b-44fc-a6f8-7c38e4f1f441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:40.847760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b3c42-bc0b-44fc-a6f8-7c38e4f1f441 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:40.848040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b3c42-bc0b-44fc-a6f8-7c38e4f1f441 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:40.848308 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b3c42-bc0b-44fc-a6f8-7c38e4f1f441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:40.857905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b3c42-bc0b-44fc-a6f8-7c38e4f1f441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:40.859330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b3c42-bc0b-44fc-a6f8-7c38e4f1f441 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:40.869044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1172/2360] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:41.250813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:41.313487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0b8ce-0971-4fff-bbb0-8bf8aada1197 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:41.317999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:41.318623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:41.318905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:41.319179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] PATH_INFO: `/v3/users/d824954f4a3c45398380c716ea7e31f9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:41.320306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d824954f4a3c45398380c716ea7e31f9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:41.322258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:41.332542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:41.368535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd824954f4a3c45398380c716ea7e31f9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.376811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd824954f4a3c45398380c716ea7e31f9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.384234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd824954f4a3c45398380c716ea7e31f9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.384903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd824954f4a3c45398380c716ea7e31f9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.398291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd824954f4a3c45398380c716ea7e31f9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.404828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1189/2361] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:07:41 2022] DELETE /identity/v3/users/d824954f4a3c45398380c716ea7e31f9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:41.409296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:41.409761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:41.409912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:41.429082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:41.430354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:41.491213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1173/2362] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:41.501182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:41.502584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:41.562906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9f98c-ab87-46e1-afe4-fe77d8926fdb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:41.566517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:41.567127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:41.567400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:41.567698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:41.576705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:41.578135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:41.587311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1190/2363] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:41.860650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:41.862226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:41.874390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d86e-fb08-4bd5-9432-d81e6ef7f2ce None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:41.879310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:41.879979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:41.880259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:41.880532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] PATH_INFO: `/v3/projects/2650c6696ab943de93e11a6489c25987` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:41.882362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2650c6696ab943de93e11a6489c25987)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:41.884336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:41.929052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:41.933276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2650c6696ab943de93e11a6489c25987', 'request_id': 'req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.939901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2650c6696ab943de93e11a6489c25987', 'request_id': 'req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.946014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2650c6696ab943de93e11a6489c25987', 'request_id': 'req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.958267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] The token cache is being invalidate because project 2650c6696ab943de93e11a6489c25987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:07:41.958559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.959176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:41.960838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1174/2364] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:07:41 2022] DELETE /identity/v3/projects/2650c6696ab943de93e11a6489c25987 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:41.964861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:41.975151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7797d7-ccc7-4ca2-9b05-f41e01f2f388 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.030782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.031407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.031723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.032002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] PATH_INFO: `/v3/users/18280a24905e432d83a1645c99056754` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.033122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18280a24905e432d83a1645c99056754)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.035076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.069558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:42.083609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18280a24905e432d83a1645c99056754'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.090754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18280a24905e432d83a1645c99056754'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.097893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18280a24905e432d83a1645c99056754'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.098520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18280a24905e432d83a1645c99056754'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.110523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40380d28-e888-47c1-b6db-64ec9dc03740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18280a24905e432d83a1645c99056754'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.116421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1191/2365] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:07:41 2022] DELETE /identity/v3/users/18280a24905e432d83a1645c99056754 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:42.386906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.388681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:42.402089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaccc6-40e7-4af4-ab5c-c16e5be5739f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.459320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.459981 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.460194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.460470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] PATH_INFO: `/v3/projects/908857250f804089bee0ed0d0b9840eb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.461608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908857250f804089bee0ed0d0b9840eb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.463555 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.510171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908857250f804089bee0ed0d0b9840eb', 'request_id': 'req-2dc72922-7de2-47fd-b6d6-e54f83effed3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.515660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908857250f804089bee0ed0d0b9840eb', 'request_id': 'req-2dc72922-7de2-47fd-b6d6-e54f83effed3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.521638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908857250f804089bee0ed0d0b9840eb', 'request_id': 'req-2dc72922-7de2-47fd-b6d6-e54f83effed3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.533947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] The token cache is being invalidate because project 908857250f804089bee0ed0d0b9840eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:07:42.534254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.534891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:07:42.536378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1175/2366] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:07:42 2022] DELETE /identity/v3/projects/908857250f804089bee0ed0d0b9840eb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:42.541192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170983e-d86e-48a2-bef6-3431ed327fba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.541536 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170983e-d86e-48a2-bef6-3431ed327fba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.541814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170983e-d86e-48a2-bef6-3431ed327fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.547937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.549363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:42.563056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6170983e-d86e-48a2-bef6-3431ed327fba None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:42.564408 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6170983e-d86e-48a2-bef6-3431ed327fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:42.611359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:42.624806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1192/2367] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:07:42.633460 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.633749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.634024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.634956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72922-7de2-47fd-b6d6-e54f83effed3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.640549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.641154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.641437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.641711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.643507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:42.654019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:42.655196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:42.662984 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1193/2368] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:07:42.667064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.677607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68f49c-fa86-46ce-b348-cbabde44b194 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.682493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.683083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.683378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.683797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.684442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:07:42.684804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.686393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.699145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1194/2369] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:07:42.703409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.707759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.709905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.713036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ede4c-b7f5-4a50-a301-66d17a6aa56e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.718494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.719086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.719211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1176/2370] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:07:42.719376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.719669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.720290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.722165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.750062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1195/2371] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:07:42 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:07:42.754477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.765683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3afd64-1e29-4161-a9bb-7e1567c762e9 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.823698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.824326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.824609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.824885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.826236 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.828301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.858698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1177/2372] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:07:42 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:42.860447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.871069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f69b-a6eb-4a2c-8a4e-9012652aac11 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.876161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.876759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.877037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.877309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.878238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.879832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.893863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1196/2373] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:42.895112 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.906319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ffb19-3434-4d46-b074-919b540609c4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.911233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16077744-2129-4839-9e85-ac5d17b920f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.911853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16077744-2129-4839-9e85-ac5d17b920f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.912156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16077744-2129-4839-9e85-ac5d17b920f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.912430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16077744-2129-4839-9e85-ac5d17b920f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.913064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16077744-2129-4839-9e85-ac5d17b920f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.914648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16077744-2129-4839-9e85-ac5d17b920f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.924926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1178/2374] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:42.929013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:42.939405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30da4-2e8f-4b92-b647-df6d6d30199a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:42.943997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:42.944637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.944917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.945186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] PATH_INFO: `/v3/projects/072ba83a2a094e38840a412b7809b4a5/users/89faead6353a4d5b92535f5113023855/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:42.960515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=072ba83a2a094e38840a412b7809b4a5, user_id=89faead6353a4d5b92535f5113023855, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:42.963100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:42.976618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1197/2375] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:42 2022] PUT /identity/v3/projects/072ba83a2a094e38840a412b7809b4a5/users/89faead6353a4d5b92535f5113023855/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:07:42.982004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bdea0-a524-48d5-9ad4-a9f907100544 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:42.982298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bdea0-a524-48d5-9ad4-a9f907100544 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:42.982574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bdea0-a524-48d5-9ad4-a9f907100544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:43.001420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b29bdea0-a524-48d5-9ad4-a9f907100544 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:43.002617 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b29bdea0-a524-48d5-9ad4-a9f907100544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:43.061423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1179/2376] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:43.070890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:43.072601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:43.108211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:43.132790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e8b6-249d-443e-8acd-03a0420edd92 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:43.136366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:43.136964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:43.137250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:43.137520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:43.146409 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:43.147921 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:43.157163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1198/2377] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:43.777550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:43.778077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:43.778209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:43.810980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] MFA Rules not processed for user `89faead6353a4d5b92535f5113023855`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:43.812281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:43.818043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:43.886817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1180/2378] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:43.891361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:43.901213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d48336-c83d-41cb-871c-9392f4e60455 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:43.907203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:43.907203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:43.919335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:43.919761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:43.920448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:43.927129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:43.965082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1199/2379] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:07:43 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:43.966631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:43.979202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12c46f-f505-4bb7-ba54-527f9170aeff None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:44.038220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:44.038836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:44.039114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:44.039385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:44.040041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:44.042007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:44.068197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1181/2380] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:07:43 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:07:44.069717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:44.080248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5972c-f933-40bd-8943-ef7f34c3fb83 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:44.089640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:44.090245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:44.090522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:44.091405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:44.092316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:44.093958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:44.105794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1200/2381] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:07:44.110790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:44.120848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0d3a8-a287-4c75-8c4f-425b96ac5738 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:44.126044 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:44.126689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:44.126966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:44.127234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] PATH_INFO: `/v3/projects/c95ad23a404a4ba08521336e3cb45070/users/a1be7b53a1f7429e8300c0384dc9ff75/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:44.142245 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95ad23a404a4ba08521336e3cb45070, user_id=a1be7b53a1f7429e8300c0384dc9ff75, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:44.144916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:44.158033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1182/2382] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:44 2022] PUT /identity/v3/projects/c95ad23a404a4ba08521336e3cb45070/users/a1be7b53a1f7429e8300c0384dc9ff75/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:44.162057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:44.172792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af3310-3f72-4a71-8c33-69404239622e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:44.229791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738f0d3-3b57-4cdd-adcd-b1c5fd9db163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:44.230320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738f0d3-3b57-4cdd-adcd-b1c5fd9db163 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:44.230599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738f0d3-3b57-4cdd-adcd-b1c5fd9db163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:44.230874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738f0d3-3b57-4cdd-adcd-b1c5fd9db163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:44.232133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738f0d3-3b57-4cdd-adcd-b1c5fd9db163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:44.233734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738f0d3-3b57-4cdd-adcd-b1c5fd9db163 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:44.243464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1201/2383] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:07:44.247887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:44.258360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f41871-347c-4f2a-8916-acc504e5b621 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:44.262965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:44.263637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:44.263922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:44.264226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] PATH_INFO: `/v3/projects/c95ad23a404a4ba08521336e3cb45070/users/a1be7b53a1f7429e8300c0384dc9ff75/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:44.266136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95ad23a404a4ba08521336e3cb45070, user_id=a1be7b53a1f7429e8300c0384dc9ff75, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:44.268830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:44.279380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1183/2384] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:07:44 2022] PUT /identity/v3/projects/c95ad23a404a4ba08521336e3cb45070/users/a1be7b53a1f7429e8300c0384dc9ff75/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:07:44.902152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:44.902442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:44.902721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:44.930789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] MFA Rules not processed for user `a1be7b53a1f7429e8300c0384dc9ff75`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:44.932064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:45.006290 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1202/2385] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:07:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:07:45.019211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:45.021094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:45.083918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f84447-c901-4bf0-812c-3c9614270c2b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:45.087516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:45.088153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:45.088430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:45.088702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:45.102241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:45.153872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:45.155314 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:45.164659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1184/2386] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:46.517928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:46.519421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:46.532414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc634e-14f7-4d1c-8193-d32d77657e15 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:46.536042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0282c1-9195-4c92-abae-509dcd84dd7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:46.536656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0282c1-9195-4c92-abae-509dcd84dd7b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:46.536932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0282c1-9195-4c92-abae-509dcd84dd7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:46.537201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0282c1-9195-4c92-abae-509dcd84dd7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:46.548001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0282c1-9195-4c92-abae-509dcd84dd7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:46.549458 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0282c1-9195-4c92-abae-509dcd84dd7b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:46.559529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1203/2387] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:51.100464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:51.101890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:51.200637 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37249ae8-94f4-4030-b704-d19b484bec3c service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:51.206681 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:51.207451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:51.208041 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:51.208333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:51.210220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:51.217731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:51.218229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:51.218354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:51.239043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:07:51.240273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:51.245658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:51.284484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:51.286697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:51.296576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1185/2388] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:07:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:07:51.310664 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1204/2389] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:07:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:07:51.363518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:07:51.365182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:07:51.428338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:07:51.446845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647009cf-fceb-4e00-b8ba-fb17fd7d7065 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:07:51.452868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:07:51.453475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:07:51.453753 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:07:51.454023 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:07:51.462069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:07:51.463692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:07:51.472198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1186/2390] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:07:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:08:00.378731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:00.442065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7012f-b0e5-40a2-bc4a-940a81327d78 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:00.446487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:00.447114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:00.447395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:00.447707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] PATH_INFO: `/v3/users/89faead6353a4d5b92535f5113023855` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:00.448819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89faead6353a4d5b92535f5113023855)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:00.451543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:00.496965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89faead6353a4d5b92535f5113023855'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.504725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89faead6353a4d5b92535f5113023855'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.511686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89faead6353a4d5b92535f5113023855'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.513034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89faead6353a4d5b92535f5113023855'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.524622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89faead6353a4d5b92535f5113023855'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.530520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1205/2391] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:00 2022] DELETE /identity/v3/users/89faead6353a4d5b92535f5113023855 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:00.706963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:00.721499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974ae91-942b-4ca7-87e4-691561bc63e4 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:00.804738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:00.805371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:00.805648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:00.805919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] PATH_INFO: `/v3/projects/072ba83a2a094e38840a412b7809b4a5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:00.807036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=072ba83a2a094e38840a412b7809b4a5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:00.809673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:00.866529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '072ba83a2a094e38840a412b7809b4a5', 'request_id': 'req-c5ef9151-0098-4dbf-af2b-53de29b4f731'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.873178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '072ba83a2a094e38840a412b7809b4a5', 'request_id': 'req-c5ef9151-0098-4dbf-af2b-53de29b4f731'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.880014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '072ba83a2a094e38840a412b7809b4a5', 'request_id': 'req-c5ef9151-0098-4dbf-af2b-53de29b4f731'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.895858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] The token cache is being invalidate because project 072ba83a2a094e38840a412b7809b4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:00.896158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.896817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:00.898317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1187/2392] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:00 2022] DELETE /identity/v3/projects/072ba83a2a094e38840a412b7809b4a5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:00.902689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:00.904904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:00.905613 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:00.906971 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:00.922603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89229245-a657-44ea-8eea-35508e642639 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:00.962459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:00.970648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef9151-0098-4dbf-af2b-53de29b4f731 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:00.976615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:00.977416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:00.977643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:00.977922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:00.979973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:00.991993 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:00.993471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:00.994122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:00.994757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:00.995036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:00.995312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] PATH_INFO: `/v3/users/a1be7b53a1f7429e8300c0384dc9ff75` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:00.997076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1be7b53a1f7429e8300c0384dc9ff75)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:00.999332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.004271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1188/2393] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:01.055548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1be7b53a1f7429e8300c0384dc9ff75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.064750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1be7b53a1f7429e8300c0384dc9ff75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.081444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1be7b53a1f7429e8300c0384dc9ff75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.082469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1be7b53a1f7429e8300c0384dc9ff75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.097607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3025ec6d-6986-46d4-95a2-d20a4926b896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1be7b53a1f7429e8300c0384dc9ff75'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.109236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1206/2394] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:00 2022] DELETE /identity/v3/users/a1be7b53a1f7429e8300c0384dc9ff75 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:01.395089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:01.397894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:01.424491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8cc7-dfc3-4625-b850-f67149b663ae service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:01.441142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:01.506788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:01.507429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.507736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.508013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] PATH_INFO: `/v3/projects/c95ad23a404a4ba08521336e3cb45070` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.509159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95ad23a404a4ba08521336e3cb45070)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:01.511088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.566010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95ad23a404a4ba08521336e3cb45070', 'request_id': 'req-b59963f1-1d4e-4a27-b17d-a15c1547219f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.571527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95ad23a404a4ba08521336e3cb45070', 'request_id': 'req-b59963f1-1d4e-4a27-b17d-a15c1547219f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.578451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95ad23a404a4ba08521336e3cb45070', 'request_id': 'req-b59963f1-1d4e-4a27-b17d-a15c1547219f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.594424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] The token cache is being invalidate because project c95ad23a404a4ba08521336e3cb45070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:01.594808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.595514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b59963f1-1d4e-4a27-b17d-a15c1547219f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:01.597123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1189/2395] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:01 2022] DELETE /identity/v3/projects/c95ad23a404a4ba08521336e3cb45070 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:01.601950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.602411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.602531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.622251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:01.623456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:01.685764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1207/2396] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:01.694080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.694491 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.694824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.716698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:01.717854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:01.725807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1190/2397] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:01.729986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:01.740278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c96053-beed-4c3d-a3cd-4f92b68b7f0c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:01.745997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:01.747283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.747566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.747879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.748528 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:01.748833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:01.750431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.762575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1208/2398] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:08:01.766552 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:01.775860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1636b97-398c-4afc-8a60-69904c701d26 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:01.780527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec526509-9830-41de-8645-554f6238707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:01.781128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec526509-9830-41de-8645-554f6238707c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.781413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec526509-9830-41de-8645-554f6238707c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.781689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec526509-9830-41de-8645-554f6238707c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.782310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec526509-9830-41de-8645-554f6238707c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:01.784259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec526509-9830-41de-8645-554f6238707c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.812111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1191/2399] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:01 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:01.816141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:01.826347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1ba0-e8ed-4c18-93e3-c983d42a5b17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:01.869311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:01.889064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:01.889672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.889948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.890221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.890857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:01.892811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.913151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1209/2400] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:01 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:01.917461 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec526509-9830-41de-8645-554f6238707c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:01.928398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec526509-9830-41de-8645-554f6238707c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:01.933018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:01.933615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.934567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.934887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.935530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:01.937157 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.947162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1192/2401] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:01.956698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:01.967468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38ee5-c015-457a-ae5b-224a73878026 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:01.972118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c85896-6887-49a8-9ab3-677dfc20622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:01.972751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c85896-6887-49a8-9ab3-677dfc20622b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:01.973031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c85896-6887-49a8-9ab3-677dfc20622b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:01.973304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c85896-6887-49a8-9ab3-677dfc20622b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:01.973976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c85896-6887-49a8-9ab3-677dfc20622b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:01.975557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c85896-6887-49a8-9ab3-677dfc20622b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:01.987137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1210/2402] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:01.991276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.002318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef4275-b69e-470f-abcb-0d8081ca59f3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.007147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.007824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.008098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.008368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] PATH_INFO: `/v3/projects/d54e3796ef3d42a290b70d263f3b94ce/users/961a2d29ca7f456f91f0eb3db2aa25b5/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.023483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54e3796ef3d42a290b70d263f3b94ce, user_id=961a2d29ca7f456f91f0eb3db2aa25b5, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.026148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.038787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1193/2403] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:01 2022] PUT /identity/v3/projects/d54e3796ef3d42a290b70d263f3b94ce/users/961a2d29ca7f456f91f0eb3db2aa25b5/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:02.049949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.050285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.050563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.081951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] MFA Rules not processed for user `961a2d29ca7f456f91f0eb3db2aa25b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:02.083153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:02.146029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1211/2404] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:02.150499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.193819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:02.217796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4532a-147d-406e-823c-3af8da5f2045 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.222568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.223182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.223464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.223779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.224408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.226293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.258615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1194/2405] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:02 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:02.259821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.271006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982673-9ed7-4915-b01e-205b4003af82 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.328889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.329580 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.329846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.330118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.330755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.332891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.356822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1212/2406] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:02 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:02.361465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.372277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6748-b6f6-4a16-a4fa-d6dc69799e77 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.377434 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.378304 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.378587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.378858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.379536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.381176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.392673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1195/2407] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:02.396657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.407219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fe007-b46f-4287-8b5b-7820efeaf8c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.411863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.412526 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.412810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.413083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] PATH_INFO: `/v3/projects/01be82c85ca64b2185a6ff2907383485/users/c4ccc3ceb379427c8d9b7e6ce9d98a42/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.428195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01be82c85ca64b2185a6ff2907383485, user_id=c4ccc3ceb379427c8d9b7e6ce9d98a42, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.430879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.445614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1213/2408] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:02 2022] PUT /identity/v3/projects/01be82c85ca64b2185a6ff2907383485/users/c4ccc3ceb379427c8d9b7e6ce9d98a42/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:02.450158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.460251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa023d-d197-49fb-b897-1893964d31f6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.518730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa97f2b-0db5-41e3-be0e-0dd55562806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.519348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa97f2b-0db5-41e3-be0e-0dd55562806b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.519661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa97f2b-0db5-41e3-be0e-0dd55562806b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.519941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa97f2b-0db5-41e3-be0e-0dd55562806b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.520593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa97f2b-0db5-41e3-be0e-0dd55562806b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.522176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa97f2b-0db5-41e3-be0e-0dd55562806b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.533162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1196/2409] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:02.537216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.548004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b11e8-b618-498c-ab16-98989c97e89f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.552603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.553247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.553521 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.553793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] PATH_INFO: `/v3/projects/01be82c85ca64b2185a6ff2907383485/users/c4ccc3ceb379427c8d9b7e6ce9d98a42/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.555697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01be82c85ca64b2185a6ff2907383485, user_id=c4ccc3ceb379427c8d9b7e6ce9d98a42, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.558444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.568673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1214/2410] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:02 2022] PUT /identity/v3/projects/01be82c85ca64b2185a6ff2907383485/users/c4ccc3ceb379427c8d9b7e6ce9d98a42/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:02.579775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeaa1d-9fdd-4ca3-acac-45cbbe354566 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.580071 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeaa1d-9fdd-4ca3-acac-45cbbe354566 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.580351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeaa1d-9fdd-4ca3-acac-45cbbe354566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.609084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-58aeaa1d-9fdd-4ca3-acac-45cbbe354566 None None] MFA Rules not processed for user `c4ccc3ceb379427c8d9b7e6ce9d98a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:02.610299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58aeaa1d-9fdd-4ca3-acac-45cbbe354566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:02.677985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1197/2411] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:02.689246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:02.690964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:02.696282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:02.750442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c80a29-594a-4a17-971c-03f45b6e0ac7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:02.754356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef3388-fef3-45ed-9bc9-e746e2036d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:02.754936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef3388-fef3-45ed-9bc9-e746e2036d57 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:02.755211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef3388-fef3-45ed-9bc9-e746e2036d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:02.755478 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef3388-fef3-45ed-9bc9-e746e2036d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:02.810562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef3388-fef3-45ed-9bc9-e746e2036d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:02.811954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef3388-fef3-45ed-9bc9-e746e2036d57 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:02.820863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1215/2412] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:04.630219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf2142-79ee-4fdc-a269-850584988444 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:04.630976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf2142-79ee-4fdc-a269-850584988444 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:04.630976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf2142-79ee-4fdc-a269-850584988444 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:04.632327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1198/2413] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:08:04.642816 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42381012-591e-41c9-838e-c7582c9fb763 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:04.643357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42381012-591e-41c9-838e-c7582c9fb763 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:04.643500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42381012-591e-41c9-838e-c7582c9fb763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:04.663790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42381012-591e-41c9-838e-c7582c9fb763 None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:04.665136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42381012-591e-41c9-838e-c7582c9fb763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:04.736543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1216/2414] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:08:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:04.753016 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf2142-79ee-4fdc-a269-850584988444 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:04.754745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ebf2142-79ee-4fdc-a269-850584988444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:04.796019 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:04.815537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf2142-79ee-4fdc-a269-850584988444 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:04.819656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87614ef9-14c8-452d-91cc-335565f9beb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:04.820239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87614ef9-14c8-452d-91cc-335565f9beb0 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:04.820526 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87614ef9-14c8-452d-91cc-335565f9beb0 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:04.820835 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87614ef9-14c8-452d-91cc-335565f9beb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:04.832578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87614ef9-14c8-452d-91cc-335565f9beb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:04.833989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87614ef9-14c8-452d-91cc-335565f9beb0 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:04.845830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1199/2415] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:04.914863 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5c34f-0fe0-4943-aefa-1682057eed5d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:04.915147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5c34f-0fe0-4943-aefa-1682057eed5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:04.915418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5c34f-0fe0-4943-aefa-1682057eed5d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:04.916839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1217/2416] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:08:04.921548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960155f-03e3-40e0-bc11-cb5205f8f9e4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:04.921841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960155f-03e3-40e0-bc11-cb5205f8f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:04.922145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960155f-03e3-40e0-bc11-cb5205f8f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:04.941808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0960155f-03e3-40e0-bc11-cb5205f8f9e4 None None] MFA Rules not processed for user `476a1946148e4758b071e9b6ce788bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:04.943004 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0960155f-03e3-40e0-bc11-cb5205f8f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:04.997724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1200/2417] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:08:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:05.009773 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5c34f-0fe0-4943-aefa-1682057eed5d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:05.011459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fee5c34f-0fe0-4943-aefa-1682057eed5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:05.025152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5c34f-0fe0-4943-aefa-1682057eed5d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:05.028705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8199e9-b9b4-436b-9203-257ae898e0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:05.029297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8199e9-b9b4-436b-9203-257ae898e0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:05.029578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8199e9-b9b4-436b-9203-257ae898e0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:05.029853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8199e9-b9b4-436b-9203-257ae898e0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:05.036805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8199e9-b9b4-436b-9203-257ae898e0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:05.038228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8199e9-b9b4-436b-9203-257ae898e0fc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:05.045999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1218/2418] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:05.128863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39358f2-515a-4d38-b606-a52958971005 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:05.129156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39358f2-515a-4d38-b606-a52958971005 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:05.129443 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39358f2-515a-4d38-b606-a52958971005 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:05.131024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1201/2419] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 9 07:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 07:08:05.136963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:05.137774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:05.138054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:05.158276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] MFA Rules not processed for user `d80a908c1da74b3fb3e82c8277556e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:05.159693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:05.170959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1219/2420] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 9 07:08:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:05.182432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39358f2-515a-4d38-b606-a52958971005 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:05.195569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39358f2-515a-4d38-b606-a52958971005 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:05.199238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:05.200469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:05.200840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:05.201117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:05.211922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:05.213387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:05.224639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1202/2421] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:05.285660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:05.287082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:05.319545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:05.358724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ef58d-eb48-4d8d-a9b8-f8cbccff205a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:05.363860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:05.364407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:05.364683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:05.364952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:05.366692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:05.379534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:05.381128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:05.391221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1220/2422] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:05.813063 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:05.817001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:05.835427 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3895-1687-41a1-b7b5-b2d09d8d35a9 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:05.850877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:05.851550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:05.851871 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:05.852151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:05.860010 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:05.983957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:05.986151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e6c3c-70b1-4086-a359-5f08ccd402b0 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:06.003778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1203/2423] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:07.142822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:07.144875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:07.157141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789b0df-e744-41d4-a5da-6a4c782ff5c5 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:07.161064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:07.161936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:07.162219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:07.162491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:07.174342 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:07.175941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:07.186529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1221/2424] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:07.202666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:07.203121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:07.203261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:07.222639 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:07.223867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:07.286051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1204/2425] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:08:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:07.298222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:07.299643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:07.384817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba28e-ba67-47b0-bb85-a2b87e846e8e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:07.391962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c44a6-adff-4223-9148-78f4519add6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:07.392626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c44a6-adff-4223-9148-78f4519add6b service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:07.392966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c44a6-adff-4223-9148-78f4519add6b service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:07.393338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c44a6-adff-4223-9148-78f4519add6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:07.405751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c44a6-adff-4223-9148-78f4519add6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:07.407548 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c44a6-adff-4223-9148-78f4519add6b service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:07.416787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1222/2426] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:08:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:08:12.439823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:12.462208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:12.511351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b525b-fa83-4d99-b12d-bed22d3acc42 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:12.516035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:12.516666 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:12.516972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:12.517243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] PATH_INFO: `/v3/users/961a2d29ca7f456f91f0eb3db2aa25b5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:12.518531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=961a2d29ca7f456f91f0eb3db2aa25b5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:12.520532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:12.577713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961a2d29ca7f456f91f0eb3db2aa25b5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:12.584931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961a2d29ca7f456f91f0eb3db2aa25b5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:12.585624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961a2d29ca7f456f91f0eb3db2aa25b5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:12.596991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961a2d29ca7f456f91f0eb3db2aa25b5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:12.611974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961a2d29ca7f456f91f0eb3db2aa25b5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:12.618432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1205/2427] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:12 2022] DELETE /identity/v3/users/961a2d29ca7f456f91f0eb3db2aa25b5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:12.623091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:12.623651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:12.623833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:12.645412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:12.646637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:12.711755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1223/2428] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:12.722444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:12.723892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:12.787153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72983-1f00-4966-aeb5-5c654f252acf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:12.791223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:12.792085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:12.792377 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:12.792654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:12.802823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:12.804300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:12.815393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1206/2429] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:13.234619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:13.235867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:13.266043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7e0e7-d5f7-4797-afc9-cdf6783f1507 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:13.278985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:13.280041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:13.280443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:13.280599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] PATH_INFO: `/v3/projects/d54e3796ef3d42a290b70d263f3b94ce` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:13.281791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54e3796ef3d42a290b70d263f3b94ce)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:13.283884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:13.325941 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:13.341102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54e3796ef3d42a290b70d263f3b94ce', 'request_id': 'req-09bed520-11e8-4630-8a43-a792121bcbb4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.349216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54e3796ef3d42a290b70d263f3b94ce', 'request_id': 'req-09bed520-11e8-4630-8a43-a792121bcbb4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.357481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54e3796ef3d42a290b70d263f3b94ce', 'request_id': 'req-09bed520-11e8-4630-8a43-a792121bcbb4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.368391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] The token cache is being invalidate because project d54e3796ef3d42a290b70d263f3b94ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:13.368822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.369315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.371052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1224/2430] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:13 2022] DELETE /identity/v3/projects/d54e3796ef3d42a290b70d263f3b94ce => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:13.374975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:13.385140 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a50f4-0063-44db-8df3-3cc6df90047c service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:13.456292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:13.476634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:13.477281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:13.477561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:13.477833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] PATH_INFO: `/v3/users/c4ccc3ceb379427c8d9b7e6ce9d98a42` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:13.478986 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ccc3ceb379427c8d9b7e6ce9d98a42)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:13.481611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:13.531906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ccc3ceb379427c8d9b7e6ce9d98a42'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.539761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ccc3ceb379427c8d9b7e6ce9d98a42'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.540470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ccc3ceb379427c8d9b7e6ce9d98a42'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.548573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ccc3ceb379427c8d9b7e6ce9d98a42'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.562829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-983192ff-96cd-4e0e-875d-b6937bcb24c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ccc3ceb379427c8d9b7e6ce9d98a42'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:13.572068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1207/2431] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:13 2022] DELETE /identity/v3/users/c4ccc3ceb379427c8d9b7e6ce9d98a42 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:13.927697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:13.927697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:13.947940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bed520-11e8-4630-8a43-a792121bcbb4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.059308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.059988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.060275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.060546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] PATH_INFO: `/v3/projects/01be82c85ca64b2185a6ff2907383485` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.061772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=01be82c85ca64b2185a6ff2907383485)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.063845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.126180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01be82c85ca64b2185a6ff2907383485', 'request_id': 'req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:14.134426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01be82c85ca64b2185a6ff2907383485', 'request_id': 'req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:14.141862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01be82c85ca64b2185a6ff2907383485', 'request_id': 'req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:14.158716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] The token cache is being invalidate because project 01be82c85ca64b2185a6ff2907383485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:14.159009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:14.159734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-eefaad84-0c1b-467b-ac6f-1aadc86bc6fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:14.161465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1225/2432] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:13 2022] DELETE /identity/v3/projects/01be82c85ca64b2185a6ff2907383485 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:14.167244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.168328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.168696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.190541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:14.191894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:14.267393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1208/2433] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:14.275403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.276159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.276438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.297565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:14.298794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:14.309081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1226/2434] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:14.313794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.325013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e738aed-9777-462e-aaf0-5e0f5526ebd0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.329877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.330592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.330980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.331359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.332182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:14.332611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.334501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.353131 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1209/2435] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:08:14.355134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.366909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045223b9-8fbe-4cec-bab4-d92225ef7680 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.378105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.378737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.379018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.379300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.379979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.381906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.430940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1227/2436] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:14 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:14.432120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.445013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bbca1-77c8-479b-a743-d92f43d9c93c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.505565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:14.522031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.523210 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.523700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.523975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.524596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.526519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.551297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1210/2437] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:14 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:14.555374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.567035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0044f-d0be-4781-9c12-4e24cfb2f038 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.572508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.573127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.573411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.573687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.574398 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.576218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.590467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1228/2438] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:14.592054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.602650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639bb8b-494c-4f2e-bf92-ab6540146ce6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.608470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d824b-89a8-4df7-96d0-ebc3a687098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.609122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d824b-89a8-4df7-96d0-ebc3a687098c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.609408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d824b-89a8-4df7-96d0-ebc3a687098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.609688 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d824b-89a8-4df7-96d0-ebc3a687098c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.610343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d824b-89a8-4df7-96d0-ebc3a687098c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.612020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d824b-89a8-4df7-96d0-ebc3a687098c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.625922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1211/2439] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:14.630453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.640740 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a34210-d5f5-4e11-9078-6933f2d209e0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.645533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.646190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.646469 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.646738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] PATH_INFO: `/v3/projects/5e73622d993e4a8fbf02d62984e02d01/users/6b594ef11ddd4492b30ae7f8cf787c35/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.649853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:14.664244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e73622d993e4a8fbf02d62984e02d01, user_id=6b594ef11ddd4492b30ae7f8cf787c35, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.667217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.685184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1229/2440] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:14 2022] PUT /identity/v3/projects/5e73622d993e4a8fbf02d62984e02d01/users/6b594ef11ddd4492b30ae7f8cf787c35/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:14.696786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.697085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.697575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.735141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] MFA Rules not processed for user `6b594ef11ddd4492b30ae7f8cf787c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:14.736560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:14.809649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1212/2441] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:14.821310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:14.823860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:14.897795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b3c8c-b3f3-425d-ad06-b6bc15e76c7a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:14.901509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0068-63e6-4fa4-a1e9-e60cd3390e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:14.902219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0068-63e6-4fa4-a1e9-e60cd3390e15 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:14.902590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0068-63e6-4fa4-a1e9-e60cd3390e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:14.902956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0068-63e6-4fa4-a1e9-e60cd3390e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:14.911929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0068-63e6-4fa4-a1e9-e60cd3390e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:14.913376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0068-63e6-4fa4-a1e9-e60cd3390e15 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:14.927952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1230/2442] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:23.143039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:23.144605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:23.226061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b01da-53d7-4710-b613-509c48d7564f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:23.231642 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:23.232247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:23.232519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:23.232787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:23.234517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:23.246254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:23.247837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:23.258165 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1213/2443] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:24.576990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:24.577511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:24.577609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:24.597467 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:24.598728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:24.659565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1231/2444] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 9 07:08:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:24.672582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:24.673991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:24.716807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:24.742273 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377eb69-562d-4a61-aae0-4ed452c0009b service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:24.747765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e4962-0fcb-4307-8e0c-1ece2b93db2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:24.748361 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4962-0fcb-4307-8e0c-1ece2b93db2b service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:24.748635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4962-0fcb-4307-8e0c-1ece2b93db2b service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:24.748902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4962-0fcb-4307-8e0c-1ece2b93db2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:24.757032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e4962-0fcb-4307-8e0c-1ece2b93db2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:24.758658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e4962-0fcb-4307-8e0c-1ece2b93db2b service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:24.766726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1214/2445] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:08:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:08:33.690856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:33.753336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f854d82-a6fc-461a-bf9e-1fbcc20aa825 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:33.757784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:33.758403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:33.758678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:33.758946 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] PATH_INFO: `/v3/users/6b594ef11ddd4492b30ae7f8cf787c35` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:33.760067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b594ef11ddd4492b30ae7f8cf787c35)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:33.762284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:33.798821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:33.810209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b594ef11ddd4492b30ae7f8cf787c35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:33.816840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b594ef11ddd4492b30ae7f8cf787c35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:33.823845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b594ef11ddd4492b30ae7f8cf787c35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:33.824488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b594ef11ddd4492b30ae7f8cf787c35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:33.836323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b594ef11ddd4492b30ae7f8cf787c35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:33.842964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1232/2446] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:33 2022] DELETE /identity/v3/users/6b594ef11ddd4492b30ae7f8cf787c35 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:33.847282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:33.847743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:33.847881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:33.877164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:33.878635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:33.938910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1215/2447] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:33.949701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:33.951325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:34.013745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b4c2c-bdf4-4296-aae9-30462f8b43bb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.017477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1934c46-f4b0-45b9-8e6f-dbee854f313b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.018084 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1934c46-f4b0-45b9-8e6f-dbee854f313b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.018366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1934c46-f4b0-45b9-8e6f-dbee854f313b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.018635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1934c46-f4b0-45b9-8e6f-dbee854f313b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.028490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1934c46-f4b0-45b9-8e6f-dbee854f313b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.029910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1934c46-f4b0-45b9-8e6f-dbee854f313b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.039894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1233/2448] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:34.307070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.308588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:34.324478 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5467106-2c84-43bb-a232-569a45688a77 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.329036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.329672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.329947 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.330355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] PATH_INFO: `/v3/projects/5e73622d993e4a8fbf02d62984e02d01` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.331346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e73622d993e4a8fbf02d62984e02d01)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.333505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.380244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e73622d993e4a8fbf02d62984e02d01', 'request_id': 'req-adb37e44-eda1-4068-88ba-2155f1043b2b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:34.386315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e73622d993e4a8fbf02d62984e02d01', 'request_id': 'req-adb37e44-eda1-4068-88ba-2155f1043b2b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:34.392518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e73622d993e4a8fbf02d62984e02d01', 'request_id': 'req-adb37e44-eda1-4068-88ba-2155f1043b2b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:34.406240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] The token cache is being invalidate because project 5e73622d993e4a8fbf02d62984e02d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:34.406533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:34.407186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-adb37e44-eda1-4068-88ba-2155f1043b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:34.408703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1216/2449] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:34 2022] DELETE /identity/v3/projects/5e73622d993e4a8fbf02d62984e02d01 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:34.413604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.413924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.414207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.433329 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:34.434535 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:34.502504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1234/2450] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:34.504200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.504521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.504832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.525055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:34.526886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:34.535834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1217/2451] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:34.539892 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.549616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a9ddc-a083-4daf-8ecf-95e2afb84b6b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.554421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.555025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.555311 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.555583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.556253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:34.556559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.558191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.569992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1235/2452] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:08:34.574852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.584303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6d95e-322e-4e27-a0cb-311a520ca82c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.589360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.590017 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.590293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.590591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.591207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.593154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.625691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1218/2453] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:34 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:34.626776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.637981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ba110-cc84-4765-be86-8f9d6a78c8c3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.677904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:34.695302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.695949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.696213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.696487 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.697104 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.699086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.719699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1236/2454] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:34 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:34.723912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.734882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb660650-007d-4860-a735-5113d7e94a00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.739533 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.740169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.740450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.740726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.741369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.742958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.753534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1219/2455] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:34.757379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.767563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4f00e-447a-4789-b479-fc8307c8ad15 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.772210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e439a-6c3c-4dee-868d-767eb64caee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.772785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e439a-6c3c-4dee-868d-767eb64caee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.773068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e439a-6c3c-4dee-868d-767eb64caee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.773344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e439a-6c3c-4dee-868d-767eb64caee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.773988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e439a-6c3c-4dee-868d-767eb64caee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.775760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e439a-6c3c-4dee-868d-767eb64caee7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.786629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1237/2456] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:34.790653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:34.801286 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc20a73-8440-40e7-ab78-0d307098ba35 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:34.805889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:34.806531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.806816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.807093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] PATH_INFO: `/v3/projects/157d70fd37d04e2bb80bba91589d919b/users/f58f2bc4c09a4b908987e04edb84cde4/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.818204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:34.822531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d70fd37d04e2bb80bba91589d919b, user_id=f58f2bc4c09a4b908987e04edb84cde4, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:34.825679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:34.838522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1220/2457] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:34 2022] PUT /identity/v3/projects/157d70fd37d04e2bb80bba91589d919b/users/f58f2bc4c09a4b908987e04edb84cde4/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:34.904077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71542-711c-4904-8718-b5292609f682 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:34.904299 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71542-711c-4904-8718-b5292609f682 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:34.904537 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71542-711c-4904-8718-b5292609f682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:34.933009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fca71542-711c-4904-8718-b5292609f682 None None] MFA Rules not processed for user `f58f2bc4c09a4b908987e04edb84cde4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:34.934237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fca71542-711c-4904-8718-b5292609f682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:34.998280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1238/2458] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:35.009578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:35.011359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:35.072274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570cdac-61d8-4034-8ac7-2468f9925abc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:35.077047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:35.077669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:35.077945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:35.078216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:35.086920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:35.088371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:35.098678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1221/2459] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:37.048136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71542-711c-4904-8718-b5292609f682 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:37.049729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fca71542-711c-4904-8718-b5292609f682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:37.173495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71542-711c-4904-8718-b5292609f682 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:37.181279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:37.182200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:37.182559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:37.183096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:37.185324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:37.200976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:37.202793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba82a1-e033-4426-8067-9cea23c9e223 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:37.217365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1239/2460] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:37.678166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:37.774425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b8917-de7d-449a-9ad0-a71ff3ed17cd service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:37.784296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:37.785271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:37.785699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:37.786122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] PATH_INFO: `/v3/users/f58f2bc4c09a4b908987e04edb84cde4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:37.787553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58f2bc4c09a4b908987e04edb84cde4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:37.793369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:37.883590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58f2bc4c09a4b908987e04edb84cde4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:37.897418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58f2bc4c09a4b908987e04edb84cde4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:37.898229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58f2bc4c09a4b908987e04edb84cde4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:37.907353 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58f2bc4c09a4b908987e04edb84cde4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:37.929787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58f2bc4c09a4b908987e04edb84cde4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:37.939693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1222/2461] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:37 2022] DELETE /identity/v3/users/f58f2bc4c09a4b908987e04edb84cde4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:37.943929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:37.944461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:37.944877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:37.970668 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:37.971955 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:38.044782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:38.089818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1240/2462] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:38.099815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:38.101665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:38.116932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:38.185667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a234707-c331-449a-b0fd-1f2e12608fcc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:38.205095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df43c2-0e3d-486b-a1a4-4bb1f6dd4b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:38.206006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df43c2-0e3d-486b-a1a4-4bb1f6dd4b09 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:38.206515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df43c2-0e3d-486b-a1a4-4bb1f6dd4b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:38.207002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df43c2-0e3d-486b-a1a4-4bb1f6dd4b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:38.218698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df43c2-0e3d-486b-a1a4-4bb1f6dd4b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:38.220712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df43c2-0e3d-486b-a1a4-4bb1f6dd4b09 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:38.233342 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1223/2463] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:38.610646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:38.612210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:38.627671 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bcbb-2095-4d66-b6f7-120c66bd4c7c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:38.636156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:38.637587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:38.638082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:38.638483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] PATH_INFO: `/v3/projects/157d70fd37d04e2bb80bba91589d919b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:38.641172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157d70fd37d04e2bb80bba91589d919b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:38.643951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:38.711541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157d70fd37d04e2bb80bba91589d919b', 'request_id': 'req-8ccde5a1-8961-4639-a222-3556e7ee0186'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:38.722543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157d70fd37d04e2bb80bba91589d919b', 'request_id': 'req-8ccde5a1-8961-4639-a222-3556e7ee0186'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:38.730918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157d70fd37d04e2bb80bba91589d919b', 'request_id': 'req-8ccde5a1-8961-4639-a222-3556e7ee0186'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:38.745888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] The token cache is being invalidate because project 157d70fd37d04e2bb80bba91589d919b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:38.746180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:38.747562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ccde5a1-8961-4639-a222-3556e7ee0186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:38.749840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1241/2464] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:38 2022] DELETE /identity/v3/projects/157d70fd37d04e2bb80bba91589d919b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:38.755211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:38.755826 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:38.756014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:38.781842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:38.783086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:38.867349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1224/2465] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:38.874865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:38.875152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:38.875477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:38.896780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:38.897981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:38.907154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1242/2466] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:38.911374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:38.921667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28b9e6-c158-4578-9526-e83f387f767f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:38.927015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:38.927649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:38.927931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:38.928199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:38.928865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:38.929151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:38.930741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:38.942941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1225/2467] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:08:38.947616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:38.957374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502217cb-7653-44fa-b313-994e8ca523ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:38.962225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:38.962841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:38.963115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:38.963431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:38.964861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:38.966760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:38.995517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1243/2468] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:38 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:38.999953 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:39.012263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c21960-604b-49a7-9f05-e7bf1b316a51 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:39.074437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:39.074970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:39.075250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:39.075521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:39.076202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:39.078145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:39.107439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1226/2469] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:38 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:39.111687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:39.126522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721527-ae37-41b0-b59f-f610f598d792 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:39.131940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:39.132779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:39.133064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:39.133340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:39.133993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:39.136454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:39.149211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1244/2470] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:39.153905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:39.164069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131c64b-67bb-44c0-b6ed-3f1da654405e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:39.168719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587adeb-db35-424f-bacf-80d1aafa0c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:39.169313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587adeb-db35-424f-bacf-80d1aafa0c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:39.169591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587adeb-db35-424f-bacf-80d1aafa0c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:39.169863 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587adeb-db35-424f-bacf-80d1aafa0c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:39.170499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587adeb-db35-424f-bacf-80d1aafa0c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:39.172123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587adeb-db35-424f-bacf-80d1aafa0c0b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:39.183748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1227/2471] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:39.188099 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:39.199199 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1faaa0-243d-49a8-89a7-b9650bbe4271 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:39.204494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:39.205164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:39.205445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:39.205721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] PATH_INFO: `/v3/projects/b65b32c9ceb64f4f9bde2d16d1df8451/users/224182ec65a94403b2a3fe8c716e9b34/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:39.221039 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65b32c9ceb64f4f9bde2d16d1df8451, user_id=224182ec65a94403b2a3fe8c716e9b34, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:39.224044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:39.237866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1245/2472] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:39 2022] PUT /identity/v3/projects/b65b32c9ceb64f4f9bde2d16d1df8451/users/224182ec65a94403b2a3fe8c716e9b34/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:39.250395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:39.250687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:39.250966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:39.283875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] MFA Rules not processed for user `224182ec65a94403b2a3fe8c716e9b34`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:39.285656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:39.294318 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:39.359150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1228/2473] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:39.371494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:39.373373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:39.433155 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f609c71-80a0-40ec-a979-6210cc6584f2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:39.436757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:39.437357 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:39.437635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:39.437906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:39.445939 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:39.447350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:39.451530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:39.459368 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1246/2474] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:42.396238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:42.397716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:42.459583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e92159-3476-4c57-9b2b-5ad9d08b7408 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:42.463945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:42.464563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:42.464853 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:42.465125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:42.466905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:42.540187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:42.542388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:42.556548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1229/2475] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:43.010693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:43.012172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:43.024296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068d51-1d8b-4f5b-aef1-428152966390 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:43.029319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:43.030050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:43.030399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:43.030744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:43.032570 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:43.045451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:43.046873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:43.056891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1247/2476] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:46.565261 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:46.566791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:46.581556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d8744-18a9-4d26-b9b4-1db121c22d19 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:46.586473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:46.587202 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:46.587490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:46.587831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] PATH_INFO: `/v3/users/4e528f53a4b94e0fab0d90e0dbe07216` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:46.589648 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e528f53a4b94e0fab0d90e0dbe07216)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:46.591683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:46.695793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e528f53a4b94e0fab0d90e0dbe07216'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:46.710374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:46.713806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e528f53a4b94e0fab0d90e0dbe07216'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:46.716167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e528f53a4b94e0fab0d90e0dbe07216'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:46.728980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e528f53a4b94e0fab0d90e0dbe07216'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:46.755522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e528f53a4b94e0fab0d90e0dbe07216'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:46.762721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1230/2477] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:46 2022] DELETE /identity/v3/users/4e528f53a4b94e0fab0d90e0dbe07216 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:47.151896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.162954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d701ea-3054-4a8e-8118-a752b165ec8f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.236889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.237558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.237838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.238114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] PATH_INFO: `/v3/projects/b7ada2fd6ffa44df9345c13afb650496` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.239310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ada2fd6ffa44df9345c13afb650496)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.241444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.250705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.252607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.269611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b03489-5952-4714-8425-74f8539bf601 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.276271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.277247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.277526 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.277969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] PATH_INFO: `/v3/users/cc885026afe041cca086367bb67d8a9b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.279110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc885026afe041cca086367bb67d8a9b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.281075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.295518 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ada2fd6ffa44df9345c13afb650496', 'request_id': 'req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.302799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ada2fd6ffa44df9345c13afb650496', 'request_id': 'req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.310038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ada2fd6ffa44df9345c13afb650496', 'request_id': 'req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.321051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] The token cache is being invalidate because project b7ada2fd6ffa44df9345c13afb650496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:47.321344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.321983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5fc7b40b-6f04-4f85-9f31-36b28f850ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.324123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1248/2478] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:47 2022] DELETE /identity/v3/projects/b7ada2fd6ffa44df9345c13afb650496 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:47.328789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.329078 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.329355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.329609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc885026afe041cca086367bb67d8a9b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.337094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc885026afe041cca086367bb67d8a9b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.337925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc885026afe041cca086367bb67d8a9b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.346860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc885026afe041cca086367bb67d8a9b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.349867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:47.351079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.359944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-07e12e2d-a928-4060-aba8-e955936a1bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc885026afe041cca086367bb67d8a9b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.365917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1231/2479] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:47 2022] DELETE /identity/v3/users/cc885026afe041cca086367bb67d8a9b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:47.411417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1249/2480] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:47.419423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.419748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.420029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.440541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:47.441764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.501978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1232/2481] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:47.506595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.518195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f8eb0-86ed-4656-a673-12954b4f6b97 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.524138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.525188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.525590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.526590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.527306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:47.527718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.529979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.539751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.541170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.547845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1250/2482] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:08:47.549680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.555759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461b2a8-3c98-4033-97b1-3d72a63ebaa5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.561533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:47.561689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.562925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.563905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.563905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] PATH_INFO: `/v3/projects/f38c6aa37b9643bbbe5bb7fd3efde4fd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.565573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38c6aa37b9643bbbe5bb7fd3efde4fd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.567195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dbde0-bac3-4c55-997a-a0950c6822c4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.568091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.573864 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.574386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.575334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.575646 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.576451 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.578496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.610515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1251/2483] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:47.611641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.612856 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=82,keystone:INSERT=2,keystone:DELETE=16 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:47.622894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38c6aa37b9643bbbe5bb7fd3efde4fd', 'request_id': 'req-52b1b2b9-a010-4a41-960f-f8e7de81bc30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.630211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38c6aa37b9643bbbe5bb7fd3efde4fd', 'request_id': 'req-52b1b2b9-a010-4a41-960f-f8e7de81bc30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.638488 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38c6aa37b9643bbbe5bb7fd3efde4fd', 'request_id': 'req-52b1b2b9-a010-4a41-960f-f8e7de81bc30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.653117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] The token cache is being invalidate because project f38c6aa37b9643bbbe5bb7fd3efde4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:47.653519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.654317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-52b1b2b9-a010-4a41-960f-f8e7de81bc30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:47.656181 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1233/2484] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:47 2022] DELETE /identity/v3/projects/f38c6aa37b9643bbbe5bb7fd3efde4fd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:47.660798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd5e86-5633-48bd-84d7-5ffdd64ccc25 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.661296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd5e86-5633-48bd-84d7-5ffdd64ccc25 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.661578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd5e86-5633-48bd-84d7-5ffdd64ccc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.680956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfdad0-4eff-48b7-9bac-5b519878bcc7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.681935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-90bd5e86-5633-48bd-84d7-5ffdd64ccc25 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:47.683451 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-90bd5e86-5633-48bd-84d7-5ffdd64ccc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.685811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.686417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.686752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.687077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.687681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.689611 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.691673 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1234/2485] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:47.700397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.700685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.700988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.712486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1252/2486] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:47.717573 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.719219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.725745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:47.726929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.732086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b850e-037f-4cd2-a178-ef2bfc0d442a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.735984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1235/2487] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:47.740034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.740307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.740920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.741202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.741477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.742126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.743806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.749739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7a69-0fb2-41e5-b460-894ea9ba06b5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.757413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1253/2488] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:47.757571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.757571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.757571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.757571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.757952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:47.757952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.758366 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.759203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.768945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d611-4b50-48f5-b5ef-153cf3baeb7f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.774230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1236/2489] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:08:47.775121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.777801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.778403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.778683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.778970 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.779641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.781231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.785364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307fa3c-5c49-416f-93f6-71ccb042b6c6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.790027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.790607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.790882 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.791161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.792710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.794279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1254/2490] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:47.794589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.798412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.808207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab9b1f-ed21-48b4-8160-e061ebbb6ea6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.817244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe383c9-eb3a-426f-9419-8517c0fe6d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.818547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe383c9-eb3a-426f-9419-8517c0fe6d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.818866 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe383c9-eb3a-426f-9419-8517c0fe6d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.819161 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe383c9-eb3a-426f-9419-8517c0fe6d11 admin admin] PATH_INFO: `/v3/projects/59b29b45804349c3b8307a56dc090b69/users/c77e15e55b204169b1426041304e368a/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.821945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1237/2491] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:47.827273 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.840064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe383c9-eb3a-426f-9419-8517c0fe6d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b29b45804349c3b8307a56dc090b69, user_id=c77e15e55b204169b1426041304e368a, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.842820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe383c9-eb3a-426f-9419-8517c0fe6d11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.856835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1255/2492] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:47 2022] PUT /identity/v3/projects/59b29b45804349c3b8307a56dc090b69/users/c77e15e55b204169b1426041304e368a/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:47.868519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.868811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.869089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.897017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] MFA Rules not processed for user `c77e15e55b204169b1426041304e368a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:47.898214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:47.898347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7f6a8-eb2a-453c-b746-0f46504b0941 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.954773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.955384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.955686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.955959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.956572 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.958475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.967330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1256/2493] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:47.968442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.982212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662670f3-dfc7-4a8d-8d4b-b31fdb68e644 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:47.986768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1238/2494] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/users => generated 302 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:47.987782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:47.990281 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:47.990992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:47.991336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:47.991789 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:47.992616 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:47.994964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:47.998520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6e2f1-323c-4442-a44d-9df6324b923a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.003978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.004709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.005045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.005704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.006024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.007629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.008310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:48.020158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1239/2495] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:48.024372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.031368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:48.034592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1257/2496] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:47 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:48.038930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692373-abe8-49dc-adb2-7c78425a4fb0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.040692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.113246 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.113890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.114168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.114439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.115107 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.116741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.121742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258100cd-14d9-4dfa-865e-bc20c286f8c4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.126503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.127492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.127806 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.128081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.128282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1240/2497] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:48.128701 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.130612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.132651 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.144186 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806216e-bc3a-44b3-9729-51eb197b8d3b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.149633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7acaa-d71a-4674-a4bb-e98cf8eeb3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.150375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7acaa-d71a-4674-a4bb-e98cf8eeb3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.150700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7acaa-d71a-4674-a4bb-e98cf8eeb3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.150977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7acaa-d71a-4674-a4bb-e98cf8eeb3ca admin admin] PATH_INFO: `/v3/projects/da20f935b3d942cc952a604e5795ca1f/users/5843648da7e042a084520744c940cc9c/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.156417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1258/2498] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:48 2022] POST /identity/v3/users => generated 308 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:48.157236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.167190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ebe27-03de-476b-9db5-8db2127ab247 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.167996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7acaa-d71a-4674-a4bb-e98cf8eeb3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=da20f935b3d942cc952a604e5795ca1f, user_id=5843648da7e042a084520744c940cc9c, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.170662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7acaa-d71a-4674-a4bb-e98cf8eeb3ca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.172111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.172787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.173065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.173339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.174065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.175778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.187172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1259/2499] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:48.188536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1241/2500] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:48 2022] PUT /identity/v3/projects/da20f935b3d942cc952a604e5795ca1f/users/5843648da7e042a084520744c940cc9c/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:48.191150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.194263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.194616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.194992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.203380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f14e1-13b8-4946-87eb-783c74f75609 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.215546 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:48.217542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:48.268081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.268844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.269184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.269522 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] PATH_INFO: `/v3/projects/8618ec4c14bf44138f00146897d8825b/users/15d42221475745dcbb16b801875b7c6d/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.288210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8618ec4c14bf44138f00146897d8825b, user_id=15d42221475745dcbb16b801875b7c6d, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.291163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.301894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1242/2501] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:48.311507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.311842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1260/2502] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:48 2022] PUT /identity/v3/projects/8618ec4c14bf44138f00146897d8825b/users/15d42221475745dcbb16b801875b7c6d/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:48.312968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:48.315964 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.375659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:48.378766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694581c5-ca0f-498b-8964-d00394857c13 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.387168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de37c6a-efea-4db5-80f6-564105919430 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.392790 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.393400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.393678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.393952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.394596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.396388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.407882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1261/2503] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:48.409908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:48.411907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.422325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba78e15-024b-41bb-9f54-6f15de7e8204 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.427330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ea0f2-e813-4aa9-b548-551d068e90f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.428235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ea0f2-e813-4aa9-b548-551d068e90f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.428524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ea0f2-e813-4aa9-b548-551d068e90f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.428803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ea0f2-e813-4aa9-b548-551d068e90f3 admin admin] PATH_INFO: `/v3/projects/8618ec4c14bf44138f00146897d8825b/users/15d42221475745dcbb16b801875b7c6d/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.430714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ea0f2-e813-4aa9-b548-551d068e90f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8618ec4c14bf44138f00146897d8825b, user_id=15d42221475745dcbb16b801875b7c6d, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.432216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.432855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.433136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.433409 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.433648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ea0f2-e813-4aa9-b548-551d068e90f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.442351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.443822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.443938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1262/2504] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:48 2022] PUT /identity/v3/projects/8618ec4c14bf44138f00146897d8825b/users/15d42221475745dcbb16b801875b7c6d/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:48.453827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1243/2505] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:48.454270 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.454563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.454902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.485054 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] MFA Rules not processed for user `15d42221475745dcbb16b801875b7c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:48.486297 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:48.574745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1263/2506] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:48 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:48.585039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.586487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:48.652644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511940-58a7-429f-bcc9-6e9c8bf1bd24 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.706170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8dd8e-ef8d-4562-89a0-6aeedb3e8fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.706876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8dd8e-ef8d-4562-89a0-6aeedb3e8fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.707222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8dd8e-ef8d-4562-89a0-6aeedb3e8fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.707444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8dd8e-ef8d-4562-89a0-6aeedb3e8fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.718162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8dd8e-ef8d-4562-89a0-6aeedb3e8fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.719680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8dd8e-ef8d-4562-89a0-6aeedb3e8fc8 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:48.730646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1244/2507] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:48 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:48.901891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:48.970744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a9ddb-08d9-4a74-8f14-9607ebaa5617 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:48.975500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:48.976282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:48.976544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:48.976814 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] PATH_INFO: `/v3/users/c77e15e55b204169b1426041304e368a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:48.977940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77e15e55b204169b1426041304e368a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:48.980520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.030591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77e15e55b204169b1426041304e368a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.038735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77e15e55b204169b1426041304e368a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.046709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77e15e55b204169b1426041304e368a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.047462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77e15e55b204169b1426041304e368a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.048598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:49.061754 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4d1af0a0-7b2f-4a74-af32-6d07e5b55ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77e15e55b204169b1426041304e368a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.068936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1264/2508] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:48 2022] DELETE /identity/v3/users/c77e15e55b204169b1426041304e368a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:49.073710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.073922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.080475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.101024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:49.102508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:49.158475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.158973 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.159090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.207882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] MFA Rules not processed for user `5843648da7e042a084520744c940cc9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:49.209802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:49.212341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1245/2509] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:49.224503 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.226749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:49.285674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:49.304331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1265/2510] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:49.308812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1109d5-5214-48e5-8263-bedbacb3c569 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.311223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.312175 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:49.312338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.313158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.313398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.313738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.326271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.326836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f53aab-c5ff-4325-b2c3-875628694538 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.329545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.332183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.332935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.333315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.333684 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.334452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.337505 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.342100 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1246/2511] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:49.370506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1266/2512] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:49 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:49.376607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.392177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade3df-d31e-4407-b872-eff2ddd8ab2e service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.472313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.473467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.473467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.473467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.474067 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.476026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.504266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1247/2513] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:49 2022] POST /identity/v3/users => generated 308 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:49.507932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.518641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e60692-b583-4715-b81e-04b8aff50fd9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.523261 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.523896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.524169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.524439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.525081 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.527023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.539696 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1267/2514] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:49.544972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.555870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3897be-4697-4518-b326-6b537a75a4f5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.561376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.562039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.562322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.562600 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] PATH_INFO: `/v3/projects/fd956b602a7643dcba34304797628905/users/0b657233dddb45b49c7234ae905a2ef6/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.580885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd956b602a7643dcba34304797628905, user_id=0b657233dddb45b49c7234ae905a2ef6, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.583517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.596898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1248/2515] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:49 2022] PUT /identity/v3/projects/fd956b602a7643dcba34304797628905/users/0b657233dddb45b49c7234ae905a2ef6/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:49.601121 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.611683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef04dd3-5e6b-4f52-86aa-55f432eebb8f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.643536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.645562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:49.689610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.690229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.690511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.690787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.691441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.693214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.703918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1268/2516] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:49.707979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.717661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c14f9-9bff-4b4c-9d39-5b396c307579 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.720456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a835087-4f0d-46e9-ab35-44f276f6f2d2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.722291 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.723475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.723795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.724073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] PATH_INFO: `/v3/projects/fd956b602a7643dcba34304797628905/users/0b657233dddb45b49c7234ae905a2ef6/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.725209 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.725892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.726071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd956b602a7643dcba34304797628905, user_id=0b657233dddb45b49c7234ae905a2ef6, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.726170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.726426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] PATH_INFO: `/v3/projects/59b29b45804349c3b8307a56dc090b69` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.727549 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b29b45804349c3b8307a56dc090b69)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.728778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.730077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:49.740831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1269/2517] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:49 2022] PUT /identity/v3/projects/fd956b602a7643dcba34304797628905/users/0b657233dddb45b49c7234ae905a2ef6/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:49.782635 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:49.785903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b29b45804349c3b8307a56dc090b69', 'request_id': 'req-958f047e-97af-4a83-8701-da0789120d19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.793292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b29b45804349c3b8307a56dc090b69', 'request_id': 'req-958f047e-97af-4a83-8701-da0789120d19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.799870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b29b45804349c3b8307a56dc090b69', 'request_id': 'req-958f047e-97af-4a83-8701-da0789120d19'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.815710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] The token cache is being invalidate because project 59b29b45804349c3b8307a56dc090b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:49.816015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.816885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:49.818510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1249/2518] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:49 2022] DELETE /identity/v3/projects/59b29b45804349c3b8307a56dc090b69 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:49.824087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:49.825926 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:49.856484 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:49.906793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5312-9748-4c36-bdc7-ceea7fc9657d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:49.969997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:49.970675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:49.970959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:49.971234 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] PATH_INFO: `/v3/users/15d42221475745dcbb16b801875b7c6d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:49.972488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d42221475745dcbb16b801875b7c6d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:49.974545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:50.022591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d42221475745dcbb16b801875b7c6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.031780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d42221475745dcbb16b801875b7c6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.040082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d42221475745dcbb16b801875b7c6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.040794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d42221475745dcbb16b801875b7c6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.058245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ed77eed-c30d-41a8-8a91-abecaed3fa4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d42221475745dcbb16b801875b7c6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.064676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1270/2519] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:49 2022] DELETE /identity/v3/users/15d42221475745dcbb16b801875b7c6d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:50.290604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:50.292108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.304573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f047e-97af-4a83-8701-da0789120d19 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:50.364380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:50.367905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.367905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.368153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] PATH_INFO: `/v3/users/224182ec65a94403b2a3fe8c716e9b34` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.373494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224182ec65a94403b2a3fe8c716e9b34)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:50.377710 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:50.440127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224182ec65a94403b2a3fe8c716e9b34'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.449408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224182ec65a94403b2a3fe8c716e9b34'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.450187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224182ec65a94403b2a3fe8c716e9b34'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.465098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224182ec65a94403b2a3fe8c716e9b34'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.485055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224182ec65a94403b2a3fe8c716e9b34'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.497978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1250/2520] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:50 2022] DELETE /identity/v3/users/224182ec65a94403b2a3fe8c716e9b34 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:50.505316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7efd9-86fb-47a1-b4cf-dd91f7b3e299 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.505958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7efd9-86fb-47a1-b4cf-dd91f7b3e299 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.505958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7efd9-86fb-47a1-b4cf-dd91f7b3e299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.527656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:50.529141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.545211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-58d7efd9-86fb-47a1-b4cf-dd91f7b3e299 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:50.546474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58d7efd9-86fb-47a1-b4cf-dd91f7b3e299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.592172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:50.596916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:50.612032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932c23c-8024-49b2-b71e-4aaa04a8b020 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:50.619030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1271/2521] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:50.621041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61547112-d5dd-421c-8ca9-31e177dcbf49 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.621391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61547112-d5dd-421c-8ca9-31e177dcbf49 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.621730 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61547112-d5dd-421c-8ca9-31e177dcbf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.621948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:50.622675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.623695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.623985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] PATH_INFO: `/v3/projects/8618ec4c14bf44138f00146897d8825b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.625481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8618ec4c14bf44138f00146897d8825b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:50.627812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:50.663324 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-61547112-d5dd-421c-8ca9-31e177dcbf49 None None] MFA Rules not processed for user `0b657233dddb45b49c7234ae905a2ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:50.664737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61547112-d5dd-421c-8ca9-31e177dcbf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.683827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8618ec4c14bf44138f00146897d8825b', 'request_id': 'req-ed595d1a-a202-4a55-9c40-14be2b0a92a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.691102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8618ec4c14bf44138f00146897d8825b', 'request_id': 'req-ed595d1a-a202-4a55-9c40-14be2b0a92a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.698793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8618ec4c14bf44138f00146897d8825b', 'request_id': 'req-ed595d1a-a202-4a55-9c40-14be2b0a92a4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.714569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] The token cache is being invalidate because project 8618ec4c14bf44138f00146897d8825b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:50.714897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.715541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:50.717631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1251/2522] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:50 2022] DELETE /identity/v3/projects/8618ec4c14bf44138f00146897d8825b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:50.718159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:50.719632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.747008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1272/2523] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:50.748448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15a2e-3503-4c31-bf82-07e0189033fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.748726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15a2e-3503-4c31-bf82-07e0189033fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.748996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15a2e-3503-4c31-bf82-07e0189033fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.775237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8af15a2e-3503-4c31-bf82-07e0189033fa None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:50.777149 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8af15a2e-3503-4c31-bf82-07e0189033fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.789111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595d1a-a202-4a55-9c40-14be2b0a92a4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:50.796593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:50.797296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.797653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.798003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.799904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.854236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1273/2524] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:50.863057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.863391 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.863798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.877331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:50.878859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:50.885877 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:50.887090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:50.890477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1252/2525] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:50.895145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1274/2526] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:50.899231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:50.908803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26eeba-1606-4030-8cc1-7657c3343953 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:50.913587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:50.914274 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.914558 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.914837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.915486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:50.915829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:50.918135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:50.930959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1253/2527] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:08:50.935963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:50.945431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ef389-3011-43dc-8c5e-185a435f8e33 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:50.950268 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:50.950904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:50.951169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:50.951438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:50.952119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:50.954028 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:50.987647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1275/2528] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:50 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:50.991276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.004571 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa6032-81c1-407b-9543-a2a13a7fe622 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.023379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:51.097371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.098159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.098550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.099337 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.099694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.101713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.131554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1254/2529] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:50 2022] POST /identity/v3/users => generated 283 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:51.136381 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.147330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130217ea-3c4e-41b9-9ba7-8fb47c139b6a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.152825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.153435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.153742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.154011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.154649 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.156273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.166495 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1276/2530] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:51.170557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.180738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428128a1-30fd-4a58-aed6-c4f27d29451a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.186557 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9b11-115a-4eee-80cc-ff79bc789bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.187163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9b11-115a-4eee-80cc-ff79bc789bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.187448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9b11-115a-4eee-80cc-ff79bc789bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.187760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9b11-115a-4eee-80cc-ff79bc789bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.188442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9b11-115a-4eee-80cc-ff79bc789bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.190052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9b11-115a-4eee-80cc-ff79bc789bf4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.201834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1255/2531] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:51.206000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.216940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38912104-5ddf-40cb-8385-719e9c397ee2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.221915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.222577 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.222857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.223132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] PATH_INFO: `/v3/projects/7255aaca520a475d9b1a1421cabdd5cb/users/3aab922bb87041deb70728cea6b9fbad/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.238945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7255aaca520a475d9b1a1421cabdd5cb, user_id=3aab922bb87041deb70728cea6b9fbad, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.241392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.255709 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1277/2532] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:51 2022] PUT /identity/v3/projects/7255aaca520a475d9b1a1421cabdd5cb/users/3aab922bb87041deb70728cea6b9fbad/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:51.268095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.268624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.269087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.298148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] MFA Rules not processed for user `3aab922bb87041deb70728cea6b9fbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:51.299373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:51.366838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1256/2533] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:51.375502 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.387617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:51.459061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a6d0a-db3e-46fc-8803-159c1473bf9a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.465076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.465432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.465699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.465986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.466269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.466903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.468073 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:51.469127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.504778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0703d-e7f6-49b7-be06-3975d59ca32b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.510725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.511368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.511712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.511930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] PATH_INFO: `/v3/projects/b65b32c9ceb64f4f9bde2d16d1df8451` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.513159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b65b32c9ceb64f4f9bde2d16d1df8451)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.517320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1278/2534] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:51 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:51.517805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.518307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.571235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b65b32c9ceb64f4f9bde2d16d1df8451', 'request_id': 'req-184af81e-3231-4288-ba86-2327e1840bb6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:51.577088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b65b32c9ceb64f4f9bde2d16d1df8451', 'request_id': 'req-184af81e-3231-4288-ba86-2327e1840bb6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:51.583873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b65b32c9ceb64f4f9bde2d16d1df8451', 'request_id': 'req-184af81e-3231-4288-ba86-2327e1840bb6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:51.594200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea339-5326-4cda-adcd-b7bc64147a62 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.599060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] The token cache is being invalidate because project b65b32c9ceb64f4f9bde2d16d1df8451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:51.599925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:51.600604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:51.602330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1257/2535] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:51 2022] DELETE /identity/v3/projects/b65b32c9ceb64f4f9bde2d16d1df8451 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:51.670807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.671453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.671784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.672064 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.672714 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.674791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.703777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1279/2536] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:51 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:51.707482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.709416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:51.723964 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af81e-3231-4288-ba86-2327e1840bb6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.730263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.730917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.731199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.731473 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.732192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.733851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.742870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:51.750256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1258/2537] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:51.755440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.767690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df74a2-a668-41e3-ad45-e7569337189d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.772746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.773641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.773923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.774214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] PATH_INFO: `/v3/projects/f2b772b107d94a38bbfe7a6077b506e0/users/227073bfa6754e71a30a2b1ada056e9a/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.793693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b772b107d94a38bbfe7a6077b506e0, user_id=227073bfa6754e71a30a2b1ada056e9a, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.796452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.812509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1280/2538] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:51 2022] PUT /identity/v3/projects/f2b772b107d94a38bbfe7a6077b506e0/users/227073bfa6754e71a30a2b1ada056e9a/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:51.816690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.833266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434b267-e986-44c2-8fe7-782bf84492f8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.904796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665d585-4c66-438e-809d-478c9e054ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.905402 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665d585-4c66-438e-809d-478c9e054ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.905692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665d585-4c66-438e-809d-478c9e054ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.905945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665d585-4c66-438e-809d-478c9e054ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.906661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665d585-4c66-438e-809d-478c9e054ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.908225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665d585-4c66-438e-809d-478c9e054ecc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.918958 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1259/2539] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:51.922900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:51.934168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb318544-40f4-4c3b-a550-3e56da3e2099 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:51.938858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:51.939550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.939862 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.940137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] PATH_INFO: `/v3/projects/f2b772b107d94a38bbfe7a6077b506e0/users/227073bfa6754e71a30a2b1ada056e9a/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:51.942196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b772b107d94a38bbfe7a6077b506e0, user_id=227073bfa6754e71a30a2b1ada056e9a, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:51.944871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:51.947308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:51.957553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1281/2540] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:51 2022] PUT /identity/v3/projects/f2b772b107d94a38bbfe7a6077b506e0/users/227073bfa6754e71a30a2b1ada056e9a/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:51.967933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:51.968227 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:51.968509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.000111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] MFA Rules not processed for user `227073bfa6754e71a30a2b1ada056e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:52.001515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.015169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.017027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.082585 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1260/2541] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:52.085735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0565334-902d-43ea-b90b-c26c08294985 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.090446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:52.091082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.091364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.091674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] PATH_INFO: `/v3/users/5843648da7e042a084520744c940cc9c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.092805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5843648da7e042a084520744c940cc9c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:52.093730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.094729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:52.095083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.146312 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5843648da7e042a084520744c940cc9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.153804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5843648da7e042a084520744c940cc9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.153995 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c745fe0-6ba9-437e-91bf-eb2c89b9d18e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.157925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:52.158568 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.158848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.159125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.161033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5843648da7e042a084520744c940cc9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.162108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5843648da7e042a084520744c940cc9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.169068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:52.170515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:52.176289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5843648da7e042a084520744c940cc9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.180376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1261/2542] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:52.186060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1282/2543] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:52 2022] DELETE /identity/v3/users/5843648da7e042a084520744c940cc9c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:52.374050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.388582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50f803-9f83-4369-873b-f09a562da329 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.413333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:52.455429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:52.456253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.456528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.456798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] PATH_INFO: `/v3/projects/da20f935b3d942cc952a604e5795ca1f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.457903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=da20f935b3d942cc952a604e5795ca1f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:52.459815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:52.508614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da20f935b3d942cc952a604e5795ca1f', 'request_id': 'req-b96868f9-867f-4135-9eed-58e7a2da8ddb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.514602 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da20f935b3d942cc952a604e5795ca1f', 'request_id': 'req-b96868f9-867f-4135-9eed-58e7a2da8ddb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.525883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da20f935b3d942cc952a604e5795ca1f', 'request_id': 'req-b96868f9-867f-4135-9eed-58e7a2da8ddb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.538739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] The token cache is being invalidate because project da20f935b3d942cc952a604e5795ca1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:52.539032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.539716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.541283 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1262/2544] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:52 2022] DELETE /identity/v3/projects/da20f935b3d942cc952a604e5795ca1f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:52.548151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.548151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.561211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d90af-9d7f-44e3-b70e-67945b6fbfc3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.612866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.614375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.625058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:52.625700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.626116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.626413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] PATH_INFO: `/v3/users/0b657233dddb45b49c7234ae905a2ef6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.627837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b657233dddb45b49c7234ae905a2ef6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:52.629759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:52.644549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:52.680601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b657233dddb45b49c7234ae905a2ef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.684438 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96868f9-867f-4135-9eed-58e7a2da8ddb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.688625 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b657233dddb45b49c7234ae905a2ef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.689359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:52.689998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.690284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.690562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] PATH_INFO: `/v3/users/3aab922bb87041deb70728cea6b9fbad` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.691742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aab922bb87041deb70728cea6b9fbad)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:52.693664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:52.696143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b657233dddb45b49c7234ae905a2ef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.696769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b657233dddb45b49c7234ae905a2ef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.709699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6671ce96-ed9a-4f31-a9ea-9f23c015cfc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b657233dddb45b49c7234ae905a2ef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.715850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1283/2545] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:52 2022] DELETE /identity/v3/users/0b657233dddb45b49c7234ae905a2ef6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:52.743317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aab922bb87041deb70728cea6b9fbad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.751170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aab922bb87041deb70728cea6b9fbad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.751898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aab922bb87041deb70728cea6b9fbad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.753878 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:52.760718 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aab922bb87041deb70728cea6b9fbad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.775908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aab922bb87041deb70728cea6b9fbad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.782013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1263/2546] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:52 2022] DELETE /identity/v3/users/3aab922bb87041deb70728cea6b9fbad => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:52.786575 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.786968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.787340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.808407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:52.809634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.884082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1284/2547] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:52.893908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.895411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.898791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:52.900142 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:52.913968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc70b-e8e3-4b8a-961a-36b896da95e9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.919780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:52.920524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:52.920870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:52.921213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] PATH_INFO: `/v3/projects/fd956b602a7643dcba34304797628905` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:52.922425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd956b602a7643dcba34304797628905)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:52.924449 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:52.970786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc28c18-0230-43c1-885b-0d0fca415fd5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:52.973169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd956b602a7643dcba34304797628905', 'request_id': 'req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.981860 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd956b602a7643dcba34304797628905', 'request_id': 'req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:52.988514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd956b602a7643dcba34304797628905', 'request_id': 'req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.000273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] The token cache is being invalidate because project fd956b602a7643dcba34304797628905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:53.000557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.001240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9fb128ca-7fcc-46d0-ad49-4592ae6a11ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.002998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1285/2548] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:52 2022] DELETE /identity/v3/projects/fd956b602a7643dcba34304797628905 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:53.008794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22a708-5e3f-443b-8c1a-ee7f48d36716 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.009075 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22a708-5e3f-443b-8c1a-ee7f48d36716 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.009350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22a708-5e3f-443b-8c1a-ee7f48d36716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.023836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.024365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.024643 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.024907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.026976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.028724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ca22a708-5e3f-443b-8c1a-ee7f48d36716 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:53.029909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ca22a708-5e3f-443b-8c1a-ee7f48d36716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.039274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:53.092462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.092978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1286/2549] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:53.093905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.101896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.102190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.102465 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.104954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1264/2550] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:53.123433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:53.124674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.133245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1287/2551] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:53.138564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.149105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d34b1-decc-46fe-8e5e-6f3176e45013 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.155672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.156281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.156554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.156858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.157573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:53.158053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.159934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.177963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1265/2552] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:08:53.178945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.188910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ded08-5ec5-4a8e-80c8-43f5ac28ca96 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.193621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.194319 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.194601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.194870 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.195497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.198097 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.226231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1288/2553] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:53.231460 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.242145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e91a96-5cd8-4403-946e-9cbd15dac583 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.270852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:53.308032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.308646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.308925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.309197 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.309829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.311793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.332672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1266/2554] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:53.336753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.348260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c3c57-5415-45d8-8ef5-9fe819ce446c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.353218 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.354559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.354832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.355105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.355844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.357430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.367704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1289/2555] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:53.372003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.382665 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc726-dcea-4a91-87db-a7ce63e2f47f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.387585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.389598 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.390024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.390654 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.390920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.391192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.391880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.393492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.404660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec59f7-3ede-4801-95c2-48b9cfcaa0e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.406315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1267/2556] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:53.410125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.410396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.411145 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.411431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.411742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] PATH_INFO: `/v3/projects/7255aaca520a475d9b1a1421cabdd5cb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.412922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7255aaca520a475d9b1a1421cabdd5cb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.414833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.422029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674b016-5c5a-497d-894f-d438fdd7d7b9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.429952 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc902e65-8ee1-496f-8a7b-63c8b66e41b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.430773 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc902e65-8ee1-496f-8a7b-63c8b66e41b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.431143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc902e65-8ee1-496f-8a7b-63c8b66e41b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.431954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc902e65-8ee1-496f-8a7b-63c8b66e41b6 admin admin] PATH_INFO: `/v3/projects/b77138338e7d42d980b2489e6329db14/users/e29ad479cca64f58a0b79cc5ca22e074/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.452235 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc902e65-8ee1-496f-8a7b-63c8b66e41b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77138338e7d42d980b2489e6329db14, user_id=e29ad479cca64f58a0b79cc5ca22e074, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.454991 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc902e65-8ee1-496f-8a7b-63c8b66e41b6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.467510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7255aaca520a475d9b1a1421cabdd5cb', 'request_id': 'req-7d439124-e1de-4b9b-8a65-80913e852aa7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.472393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1268/2557] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:53 2022] PUT /identity/v3/projects/b77138338e7d42d980b2489e6329db14/users/e29ad479cca64f58a0b79cc5ca22e074/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:53.475749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7255aaca520a475d9b1a1421cabdd5cb', 'request_id': 'req-7d439124-e1de-4b9b-8a65-80913e852aa7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.482817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994975bd-83fc-4de5-9454-18072490f006 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.483104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994975bd-83fc-4de5-9454-18072490f006 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.483210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7255aaca520a475d9b1a1421cabdd5cb', 'request_id': 'req-7d439124-e1de-4b9b-8a65-80913e852aa7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.483383 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994975bd-83fc-4de5-9454-18072490f006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.493690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] The token cache is being invalidate because project 7255aaca520a475d9b1a1421cabdd5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:53.493977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.494876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.496418 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1290/2558] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:53 2022] DELETE /identity/v3/projects/7255aaca520a475d9b1a1421cabdd5cb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:53.500172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.501501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.514037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-994975bd-83fc-4de5-9454-18072490f006 None None] MFA Rules not processed for user `e29ad479cca64f58a0b79cc5ca22e074`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:53.515424 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-994975bd-83fc-4de5-9454-18072490f006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.568474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439124-e1de-4b9b-8a65-80913e852aa7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.572659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:53.580343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1269/2559] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:53.584840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994975bd-83fc-4de5-9454-18072490f006 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.586289 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-994975bd-83fc-4de5-9454-18072490f006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.657162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994975bd-83fc-4de5-9454-18072490f006 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.662130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.662765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.663053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.663324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.663997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.665998 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.682968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:53.700842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1270/2560] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:53.701982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.767706 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.768496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.768776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.769037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] PATH_INFO: `/v3/users/227073bfa6754e71a30a2b1ada056e9a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.770167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=227073bfa6754e71a30a2b1ada056e9a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.772188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.773697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2179d0-249d-4421-808c-2707886d9560 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.779338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.779978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.780258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.780531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.781148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.783053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.804285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1271/2561] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:53 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:53.809068 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.818550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b84e8-071f-475b-ae4f-3188f71cfff1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.822402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227073bfa6754e71a30a2b1ada056e9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.823299 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.823910 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.824194 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.824467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.825108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.826706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.830356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227073bfa6754e71a30a2b1ada056e9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.837453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227073bfa6754e71a30a2b1ada056e9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.839119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227073bfa6754e71a30a2b1ada056e9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.839470 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1272/2562] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:53.843218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.845090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:53.858977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22554552-1ad0-4e2f-8722-af46459fe0cb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:53.859677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227073bfa6754e71a30a2b1ada056e9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:53.863687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:53.864468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:53.864750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:53.865020 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] PATH_INFO: `/v3/projects/b828e0247bc84a608506fe993b30e36f/users/f7085038c36442ccbf5cf2c80ba188da/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:53.866111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1291/2563] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:53 2022] DELETE /identity/v3/users/227073bfa6754e71a30a2b1ada056e9a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:53.881109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b828e0247bc84a608506fe993b30e36f, user_id=f7085038c36442ccbf5cf2c80ba188da, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:53.883786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:53.898479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1273/2564] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:53 2022] PUT /identity/v3/projects/b828e0247bc84a608506fe993b30e36f/users/f7085038c36442ccbf5cf2c80ba188da/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:53.902542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:53.976464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784f7a1-741e-49e4-a6b7-25cba3de1801 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.006013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:54.037694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab34d8-4755-4fa3-8d3f-7c05aa95b963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.038506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab34d8-4755-4fa3-8d3f-7c05aa95b963 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.039729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab34d8-4755-4fa3-8d3f-7c05aa95b963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.040008 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab34d8-4755-4fa3-8d3f-7c05aa95b963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.040660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab34d8-4755-4fa3-8d3f-7c05aa95b963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.042534 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab34d8-4755-4fa3-8d3f-7c05aa95b963 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.055643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1292/2565] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:54.058928 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.069355 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc138-9db7-4ff3-8ae2-8c404f46822e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.074968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.075729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.076087 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.076406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] PATH_INFO: `/v3/projects/b828e0247bc84a608506fe993b30e36f/users/f7085038c36442ccbf5cf2c80ba188da/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.078382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b828e0247bc84a608506fe993b30e36f, user_id=f7085038c36442ccbf5cf2c80ba188da, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.081108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.092447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1274/2566] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:54 2022] PUT /identity/v3/projects/b828e0247bc84a608506fe993b30e36f/users/f7085038c36442ccbf5cf2c80ba188da/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:54.104031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.104320 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.104600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.136698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] MFA Rules not processed for user `f7085038c36442ccbf5cf2c80ba188da`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:54.138015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.151612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.153499 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.216024 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1293/2567] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:54.226781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.228151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.237570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbca9de-aa44-49f0-9491-514e83086164 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.242221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.242850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.243128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.243432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] PATH_INFO: `/v3/projects/f2b772b107d94a38bbfe7a6077b506e0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.245221 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b772b107d94a38bbfe7a6077b506e0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.247147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.286844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853436cf-0bc1-43e8-b5ee-43ca5ba6a488 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.290143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:54.298661 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b772b107d94a38bbfe7a6077b506e0', 'request_id': 'req-5a7caa28-93aa-495e-bdce-6443894d61f9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:54.304860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b772b107d94a38bbfe7a6077b506e0', 'request_id': 'req-5a7caa28-93aa-495e-bdce-6443894d61f9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:54.311126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b772b107d94a38bbfe7a6077b506e0', 'request_id': 'req-5a7caa28-93aa-495e-bdce-6443894d61f9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:54.323752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] The token cache is being invalidate because project f2b772b107d94a38bbfe7a6077b506e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:54.323975 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:54.324624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a7caa28-93aa-495e-bdce-6443894d61f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:54.326123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1275/2568] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:54 2022] DELETE /identity/v3/projects/f2b772b107d94a38bbfe7a6077b506e0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:54.330731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3734355-3a67-4ff3-934b-1de94631d543 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.331013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3734355-3a67-4ff3-934b-1de94631d543 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.331293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3734355-3a67-4ff3-934b-1de94631d543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.337259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.338459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.338847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.339211 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.341732 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.351450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f3734355-3a67-4ff3-934b-1de94631d543 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:54.352677 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f3734355-3a67-4ff3-934b-1de94631d543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.366792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:54.407317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.408793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.413649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1276/2569] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:54.419043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1294/2570] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:54.421614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.421907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.422182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.443276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:54.445190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.455485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1277/2571] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:54.459777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.471302 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3a2ce-86a8-4ab8-8a74-a747809a2272 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.476036 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.476630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.476905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.477170 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.477805 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:54.478101 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.480226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.495661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1295/2572] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:08:54.497752 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.507399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a952eb-7ae2-4350-ad56-2ce7d9da6c2a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.512858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.513537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.513827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.514160 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.514837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.517034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.551949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1278/2573] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:54.553148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.564511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84434-c259-42d8-b900-fb2430ca1011 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.626767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.627661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.628040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.628345 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.628967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.630879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.654612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1296/2574] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:54.658769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.669950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02450ef5-60be-45ee-b840-4c937477e1c7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.675057 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.675823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.676178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.676462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.677110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.678689 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.690357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1279/2575] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:54.694630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.707908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc7baf-f449-40b0-bd1f-1b386fafbab6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.711777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8ab05-5d44-427d-ae25-e52834380fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.712387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ab05-5d44-427d-ae25-e52834380fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.712710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ab05-5d44-427d-ae25-e52834380fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.712983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8ab05-5d44-427d-ae25-e52834380fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.713761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8ab05-5d44-427d-ae25-e52834380fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.715349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8ab05-5d44-427d-ae25-e52834380fa4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.726803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1297/2576] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:54.730977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.741269 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad56d04-5180-4f16-844a-b32d48f10331 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.746713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.747313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.747622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.747902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] PATH_INFO: `/v3/projects/e53e57d8fb3e4a8fb1595ffff40d3069/users/a09f87d397f444cb905c90e7c167aa4a/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.764059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53e57d8fb3e4a8fb1595ffff40d3069, user_id=a09f87d397f444cb905c90e7c167aa4a, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.766870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.780875 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1280/2577] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:54 2022] PUT /identity/v3/projects/e53e57d8fb3e4a8fb1595ffff40d3069/users/a09f87d397f444cb905c90e7c167aa4a/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:54.792153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.792490 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.792774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.821524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] MFA Rules not processed for user `a09f87d397f444cb905c90e7c167aa4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:54.823566 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:54.901394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1298/2578] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:54.906156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:54.976013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba3c0-f37c-4496-b6e9-b51879ab3fb6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:54.981513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:54.982122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:54.982399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:54.982669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:54.983371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:54.985462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:54.995694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:55.019706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1281/2579] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:54 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:55.021240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.032755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33991b-a8e5-42f8-a2d1-f4dd6c8d5422 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.044820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:55.093950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.094595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.094881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.095207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.095818 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.097715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.126022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1299/2580] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:55 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:55.126893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.138587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bfd57-5063-4954-873f-5a7a20573e23 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.146608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.147218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.147524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.147890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.148530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.150849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.164815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1282/2581] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:55.166273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.177117 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d942cde-eef7-4886-8b99-f8d9a69cef49 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.183106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.183802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.184087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.184429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] PATH_INFO: `/v3/projects/9ef8957238594d2e801d7ffd180f2565/users/d00827ad93264151929fb01c35e51931/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.205200 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8957238594d2e801d7ffd180f2565, user_id=d00827ad93264151929fb01c35e51931, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.210182 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.228988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1300/2582] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:55 2022] PUT /identity/v3/projects/9ef8957238594d2e801d7ffd180f2565/users/d00827ad93264151929fb01c35e51931/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:55.234344 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.246178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8e942-d0b1-443a-a914-08466edb25c1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.314847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd8379-d0c1-4e13-a88e-b0d4a55ca369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.315464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd8379-d0c1-4e13-a88e-b0d4a55ca369 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.315790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd8379-d0c1-4e13-a88e-b0d4a55ca369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.316070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd8379-d0c1-4e13-a88e-b0d4a55ca369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.316716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd8379-d0c1-4e13-a88e-b0d4a55ca369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.318360 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd8379-d0c1-4e13-a88e-b0d4a55ca369 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.335481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1283/2583] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:55.336500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.347990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45da7a-0b12-4623-9cd5-0e61bf4ddbbb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.352756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.353554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.353962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.354242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] PATH_INFO: `/v3/projects/9ef8957238594d2e801d7ffd180f2565/users/d00827ad93264151929fb01c35e51931/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.356222 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8957238594d2e801d7ffd180f2565, user_id=d00827ad93264151929fb01c35e51931, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.358824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.370129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1301/2584] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:55 2022] PUT /identity/v3/projects/9ef8957238594d2e801d7ffd180f2565/users/d00827ad93264151929fb01c35e51931/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:55.380819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.381105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.381385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.414793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] MFA Rules not processed for user `d00827ad93264151929fb01c35e51931`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:55.416047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:55.488035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1284/2585] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:55.500060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.501779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:55.570728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8b761-336e-4cf7-82ba-16285620b283 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.575051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e60e8f-f52e-4bee-871d-46632ae65fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.575948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e60e8f-f52e-4bee-871d-46632ae65fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.576229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e60e8f-f52e-4bee-871d-46632ae65fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.576503 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e60e8f-f52e-4bee-871d-46632ae65fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.585405 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e60e8f-f52e-4bee-871d-46632ae65fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.586823 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e60e8f-f52e-4bee-871d-46632ae65fc7 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.597112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1302/2586] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:55.626122 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.695305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818428a-2aef-46b4-98b2-06024fa4dd98 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.700778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.701415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.701699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.701970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] PATH_INFO: `/v3/users/a09f87d397f444cb905c90e7c167aa4a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.703106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09f87d397f444cb905c90e7c167aa4a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.705048 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.706644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:55.762516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09f87d397f444cb905c90e7c167aa4a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.769447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09f87d397f444cb905c90e7c167aa4a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.770147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09f87d397f444cb905c90e7c167aa4a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.779684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09f87d397f444cb905c90e7c167aa4a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.792905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09f87d397f444cb905c90e7c167aa4a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.799629 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1285/2587] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:55 2022] DELETE /identity/v3/users/a09f87d397f444cb905c90e7c167aa4a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:55.806340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.807558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.807558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.823214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.824668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:55.828375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:55.829657 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:55.893864 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae1991-9db2-44da-afe8-9e4c75b0be49 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.896802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1303/2588] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:55.898719 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.899357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.899664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.899943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] PATH_INFO: `/v3/users/e29ad479cca64f58a0b79cc5ca22e074` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.901206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29ad479cca64f58a0b79cc5ca22e074)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.903240 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.906597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:55.907960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:55.921695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:55.952614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29ad479cca64f58a0b79cc5ca22e074'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.959724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29ad479cca64f58a0b79cc5ca22e074'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.960397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29ad479cca64f58a0b79cc5ca22e074'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.968151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29ad479cca64f58a0b79cc5ca22e074'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.970829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77539059-eb24-45c9-9c18-7b407211e520 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:55.974594 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:55.975267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.975636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.975979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:55.977780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:55.981225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0419e594-7e16-43c6-b3f9-eeed7820ebeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29ad479cca64f58a0b79cc5ca22e074'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:55.982664 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:55.988902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1286/2589] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:55 2022] DELETE /identity/v3/users/e29ad479cca64f58a0b79cc5ca22e074 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:55.991549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:55.993203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:55.993354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:55.993906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:55.994936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:56.003993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1304/2590] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:56.015159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:56.016411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:56.081142 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1287/2591] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:56.092258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:56.093719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:56.105399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c10c8-86ab-4f56-8500-54fb86ec02bb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:56.158072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:56.158686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:56.158962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:56.159229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:56.170162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:56.173440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:56.183459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1305/2592] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:56.592061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:56.593515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:56.605985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c74452-f1d4-4075-a731-8d10ffa3128c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:56.610514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:56.611145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:56.611417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:56.611724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] PATH_INFO: `/v3/projects/e53e57d8fb3e4a8fb1595ffff40d3069` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:56.612845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53e57d8fb3e4a8fb1595ffff40d3069)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:56.614763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:56.672386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53e57d8fb3e4a8fb1595ffff40d3069', 'request_id': 'req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.678094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53e57d8fb3e4a8fb1595ffff40d3069', 'request_id': 'req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.685084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53e57d8fb3e4a8fb1595ffff40d3069', 'request_id': 'req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.697251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] The token cache is being invalidate because project e53e57d8fb3e4a8fb1595ffff40d3069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:56.697842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.698296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.702786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1288/2593] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:56 2022] DELETE /identity/v3/projects/e53e57d8fb3e4a8fb1595ffff40d3069 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:56.704112 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:56.715068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7af0f-51b7-43c3-9c72-a3b3b404eddf service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:56.776636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:56.777304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:56.777586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:56.777861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] PATH_INFO: `/v3/users/d00827ad93264151929fb01c35e51931` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:56.778986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d00827ad93264151929fb01c35e51931)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:56.780847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:56.780991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:56.782716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:56.797896 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d7d79-85a5-44ef-93fd-a07d31c1d042 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:56.803530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:56.804222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:56.804645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:56.804926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] PATH_INFO: `/v3/projects/b77138338e7d42d980b2489e6329db14` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:56.806089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77138338e7d42d980b2489e6329db14)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:56.809674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:56.835337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd00827ad93264151929fb01c35e51931'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.849235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd00827ad93264151929fb01c35e51931'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.855278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:56.858635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd00827ad93264151929fb01c35e51931'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.859660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd00827ad93264151929fb01c35e51931'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.870185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77138338e7d42d980b2489e6329db14', 'request_id': 'req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.874976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd00827ad93264151929fb01c35e51931'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.876161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77138338e7d42d980b2489e6329db14', 'request_id': 'req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.881371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1306/2594] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:56 2022] DELETE /identity/v3/users/d00827ad93264151929fb01c35e51931 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:56.886218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77138338e7d42d980b2489e6329db14', 'request_id': 'req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.902244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] The token cache is being invalidate because project b77138338e7d42d980b2489e6329db14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:56.902539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.903190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:56.904836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1289/2595] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:56 2022] DELETE /identity/v3/projects/b77138338e7d42d980b2489e6329db14 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:56.908736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:56.910108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:56.979343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1c65-9449-4d16-8fc6-fc9624d9248d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:56.983896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:56.984514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:56.984793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:56.985063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] PATH_INFO: `/v3/users/f7085038c36442ccbf5cf2c80ba188da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:56.986188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7085038c36442ccbf5cf2c80ba188da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:56.988148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.036555 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7085038c36442ccbf5cf2c80ba188da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.045956 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7085038c36442ccbf5cf2c80ba188da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.055068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7085038c36442ccbf5cf2c80ba188da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.055932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7085038c36442ccbf5cf2c80ba188da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.068523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7085038c36442ccbf5cf2c80ba188da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.075111 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1307/2596] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:08:56 2022] DELETE /identity/v3/users/f7085038c36442ccbf5cf2c80ba188da => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:57.235886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.237393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:57.245727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:57.333486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdabbb4-e1ee-48f2-a788-58e0c5ac5fe5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.392234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.392873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.393156 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.393464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] PATH_INFO: `/v3/projects/9ef8957238594d2e801d7ffd180f2565` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.394598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef8957238594d2e801d7ffd180f2565)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.396541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.424298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.426298 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:57.443157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead637d-c8c8-4d1a-8e54-ec4385bfd58a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.460129 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef8957238594d2e801d7ffd180f2565', 'request_id': 'req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.467047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef8957238594d2e801d7ffd180f2565', 'request_id': 'req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.473437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef8957238594d2e801d7ffd180f2565', 'request_id': 'req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.486371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] The token cache is being invalidate because project 9ef8957238594d2e801d7ffd180f2565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:57.486588 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.487201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cbaa50c1-5a39-4518-9df9-8746fcd4a8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.488736 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1290/2597] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:57 2022] DELETE /identity/v3/projects/9ef8957238594d2e801d7ffd180f2565 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:57.493389 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242895c-2e3c-4d62-8bd5-a5b6503db874 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.493672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242895c-2e3c-4d62-8bd5-a5b6503db874 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.494003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242895c-2e3c-4d62-8bd5-a5b6503db874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.497827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:57.506195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.506952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.507435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.507891 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] PATH_INFO: `/v3/projects/b828e0247bc84a608506fe993b30e36f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.509206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b828e0247bc84a608506fe993b30e36f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.511859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.514615 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5242895c-2e3c-4d62-8bd5-a5b6503db874 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:57.515857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5242895c-2e3c-4d62-8bd5-a5b6503db874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:57.523589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1291/2598] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:57.531812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.532095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.532368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.552645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:57.554058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:57.563678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b828e0247bc84a608506fe993b30e36f', 'request_id': 'req-5541f04d-d54f-4e3a-a26f-b237595d3b23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.571579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b828e0247bc84a608506fe993b30e36f', 'request_id': 'req-5541f04d-d54f-4e3a-a26f-b237595d3b23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.578658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b828e0247bc84a608506fe993b30e36f', 'request_id': 'req-5541f04d-d54f-4e3a-a26f-b237595d3b23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.586108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:57.589233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] The token cache is being invalidate because project b828e0247bc84a608506fe993b30e36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:08:57.589488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.591124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5541f04d-d54f-4e3a-a26f-b237595d3b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:08:57.591650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1308/2599] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:08:57 2022] DELETE /identity/v3/projects/b828e0247bc84a608506fe993b30e36f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:57.596836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83bd1c-0aa2-4a4b-83a6-3135d17d937b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.597120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83bd1c-0aa2-4a4b-83a6-3135d17d937b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.597397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83bd1c-0aa2-4a4b-83a6-3135d17d937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.616243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ce83bd1c-0aa2-4a4b-83a6-3135d17d937b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:57.617148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1292/2600] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:57.617456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce83bd1c-0aa2-4a4b-83a6-3135d17d937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:57.622164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.625990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1309/2601] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:57.631985 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0da44-039b-4716-9fdf-ade97eeb6735 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.633620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.633903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.634174 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.637609 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.638219 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.638504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.638776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.639428 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:57.639788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.641381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.654612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1293/2602] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:08:57.655867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:57.657059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:57.658973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.666922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1310/2603] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:57.668801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f60f9-1ed2-48ed-8a04-5b7272e97050 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.670640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.673778 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.674374 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.674656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.674933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.675584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.677509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.680617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60bac7-cad0-42cf-bfe9-ee37fc2e4688 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.686259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.686874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.687132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.687403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.688089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:08:57.688387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.690011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.703048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1311/2604] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:08:57.707340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.709225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1294/2605] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:57.713822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.718198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eac9e-1492-4654-af25-dbe88e3438cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.776335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada260-9fed-452e-b50d-58d4afbd4f38 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.777157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.777778 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.778076 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.778328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.778949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.780867 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.780996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.781601 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.781886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.782162 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.782790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.784722 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.807189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1295/2606] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:57.811628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1312/2607] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:57.813464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.813591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.823398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0b13-d403-4d06-bc15-863efe67dc2e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.865128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:57.883340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2df76-2e40-475b-8a61-3efbf9d82e58 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.888230 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.888370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.888865 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.889150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.889425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.889536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.889881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.890046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.890338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.891098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.891994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.892828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.903581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1296/2608] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:57.907364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.913241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1313/2609] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:57 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:57.917638 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.918358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b11c5e-7502-44c7-9da3-4590e0ce3e03 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.922968 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.923567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.923880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.924154 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.924788 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.926371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.927055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c986c2e-4494-4394-8e63-ab10fd44e759 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.931858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.932443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.932725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.932999 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.933641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.935331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.936670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1297/2610] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:57.940702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.948901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1314/2611] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:57.950128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.951002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62628c-fa54-4dee-b0dd-59703f2c5dfc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.956076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc99383-ba7f-4906-893d-6fdbb4f33531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.956730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc99383-ba7f-4906-893d-6fdbb4f33531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.957011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc99383-ba7f-4906-893d-6fdbb4f33531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.957288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc99383-ba7f-4906-893d-6fdbb4f33531 admin admin] PATH_INFO: `/v3/projects/061b02a6c1474f0283b260178f849576/users/250596ccca06425d90511eadd260acf5/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.959850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85570f9f-c910-4383-a0a9-32c2b9186085 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:57.966597 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:57.967193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:57.967482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:57.967791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:57.968434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.970012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.971284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:57.975241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc99383-ba7f-4906-893d-6fdbb4f33531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061b02a6c1474f0283b260178f849576, user_id=250596ccca06425d90511eadd260acf5, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:57.978505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc99383-ba7f-4906-893d-6fdbb4f33531 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:57.981561 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1315/2612] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:57.986336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:57.992034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1298/2613] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:57 2022] PUT /identity/v3/projects/061b02a6c1474f0283b260178f849576/users/250596ccca06425d90511eadd260acf5/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:58.001562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2821afb-eee8-40b4-b200-e4d9fffe5e98 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.003266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.003548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.003851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.032936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] MFA Rules not processed for user `250596ccca06425d90511eadd260acf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:58.034125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:58.053464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61750a11-1af5-4d94-bf56-d2d87dd75da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.054129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61750a11-1af5-4d94-bf56-d2d87dd75da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.054411 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61750a11-1af5-4d94-bf56-d2d87dd75da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.054683 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61750a11-1af5-4d94-bf56-d2d87dd75da2 admin admin] PATH_INFO: `/v3/projects/bb64db740d0b493db58e67344d740823/users/9902bfe1b82747c5be96717ad9bceffe/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.070841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61750a11-1af5-4d94-bf56-d2d87dd75da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb64db740d0b493db58e67344d740823, user_id=9902bfe1b82747c5be96717ad9bceffe, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.073715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61750a11-1af5-4d94-bf56-d2d87dd75da2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.086379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1316/2614] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:57 2022] PUT /identity/v3/projects/bb64db740d0b493db58e67344d740823/users/9902bfe1b82747c5be96717ad9bceffe/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:58.096820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.097103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.097448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.102775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1299/2615] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:58.107545 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.117809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6c3f4-70a9-4468-9adf-f32a9581e645 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.126775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] MFA Rules not processed for user `9902bfe1b82747c5be96717ad9bceffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:58.128292 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:58.176494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.177287 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.177663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.177997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.178616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.180548 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.195845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1317/2616] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:58.202658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.212459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1300/2617] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.214105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.214380 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53896fe6-b7a4-4cc3-bb2e-13ae37540664 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.238185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:58.278757 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.278978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98181a-999f-4aca-87b1-e5f4ba216413 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.279371 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.279687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.279965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.280583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.282472 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.284047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.284598 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.284876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.285145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.285749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.287720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.300381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:58.311807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1318/2618] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.315889 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1301/2619] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:08:58.316025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.317171 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.327188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21529-47d9-4cf0-977c-270a1ac127f2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.377399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3af515-e1bc-4138-ac2f-3d610149e704 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.382148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.382760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.383043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.383333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.383965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.385687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.386404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.386596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.386708 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.386940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.387581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.389220 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.399407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1302/2620] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:58.403645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.408583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1319/2621] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.412103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.413312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c3e3-2d89-48b3-a68b-d17a5b827ed0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.417930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.418553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.418830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.419098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.419762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.421321 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.421568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e560e72-5009-4f3b-abcb-ed40488c9e01 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.426672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.426861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.427143 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.427414 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.428086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.429650 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.435440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1303/2622] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:58.436255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.441492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1320/2623] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:58.445278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.445869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4bd5f-ab0b-41df-8932-a6dcb3ad96a0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.451217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa51bd-ebb8-4660-849c-c872a842008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.451885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa51bd-ebb8-4660-849c-c872a842008b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.452164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa51bd-ebb8-4660-849c-c872a842008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.452433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa51bd-ebb8-4660-849c-c872a842008b admin admin] PATH_INFO: `/v3/projects/34143bec062b4a2aa91195b41ceebd2c/users/feefd199c59f461c8a97bad2f3bdb086/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.456050 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45464186-c754-4cbd-9bce-971814daa4f9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.460168 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.460756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.461041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.461318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.461949 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.463621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.468381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa51bd-ebb8-4660-849c-c872a842008b admin admin] RBAC: Authorizing `identity:create_grant(project_id=34143bec062b4a2aa91195b41ceebd2c, user_id=feefd199c59f461c8a97bad2f3bdb086, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.471080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa51bd-ebb8-4660-849c-c872a842008b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.473772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1321/2624] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:58.478138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.484121 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1304/2625] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:58 2022] PUT /identity/v3/projects/34143bec062b4a2aa91195b41ceebd2c/users/feefd199c59f461c8a97bad2f3bdb086/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:58.487881 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669872-6b3d-42ba-88d9-90db0a7bcdb1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.494561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.494945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.495217 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.522394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] MFA Rules not processed for user `feefd199c59f461c8a97bad2f3bdb086`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:58.523705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:58.547254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77928d6e-bcc2-4360-9ab1-e80d7a70fb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.547861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77928d6e-bcc2-4360-9ab1-e80d7a70fb4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.548138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77928d6e-bcc2-4360-9ab1-e80d7a70fb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.548410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77928d6e-bcc2-4360-9ab1-e80d7a70fb4e admin admin] PATH_INFO: `/v3/projects/fcb0c96e7ac9484c9b7dc5602166630f/users/f19925dbf5ed441a886c8f4693bbca10/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.563512 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77928d6e-bcc2-4360-9ab1-e80d7a70fb4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb0c96e7ac9484c9b7dc5602166630f, user_id=f19925dbf5ed441a886c8f4693bbca10, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.566141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77928d6e-bcc2-4360-9ab1-e80d7a70fb4e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.576262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:58.581607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1322/2626] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:58 2022] PUT /identity/v3/projects/fcb0c96e7ac9484c9b7dc5602166630f/users/f19925dbf5ed441a886c8f4693bbca10/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:58.593272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.593557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.593966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.601036 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1305/2627] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:58.605951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.616203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e4107-957c-4b41-8e87-ded814cdcbc9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.624260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] MFA Rules not processed for user `f19925dbf5ed441a886c8f4693bbca10`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:58.625481 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:58.668357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:58.684207 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.684912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.685199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.685475 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.686117 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.688094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.694383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1323/2628] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:58.699511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.709693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2705b-7ad2-47c1-adc6-32a5fcf000fa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.714943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.715545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.715859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.716135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.716762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.717172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1306/2629] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.720999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.721236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.747760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1324/2630] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.751772 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.786510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd869-44aa-4b5b-9f11-eb1ab33c9543 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.816723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9302f-2814-4910-bc69-0c5cebe22eaa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.821304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.822448 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.822726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.823043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.823736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.825635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.841692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.842511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.842892 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.843169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.844029 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.846065 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.851932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1325/2631] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.857900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.869919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5848-6acc-4365-9646-d25064107988 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.875538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1307/2632] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:08:58 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:08:58.875717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.875717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.875987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.876271 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.876924 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.877312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.878775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.888077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488052ed-052e-4874-b894-5fa38d6c4f7e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.892474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1326/2633] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:58.893415 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.896214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.896850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.897132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.897407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.898061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.899670 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.903057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8df6a-eb23-4822-b558-4db8f4f56508 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.907745 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.908410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.908691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.908965 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] PATH_INFO: `/v3/projects/d0024519726d4bc5a73c996909d68604/users/8670d0b5debd4fa49ebdf566c2b4b7d4/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.909825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1308/2634] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:08:58.913870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.923967 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d265d0-fceb-4ed7-b936-5a489597f491 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:58.928051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0024519726d4bc5a73c996909d68604, user_id=8670d0b5debd4fa49ebdf566c2b4b7d4, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.928692 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:58.929363 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:58.929659 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:58.929939 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] PATH_INFO: `/v3/projects/6883f78ae4c94b6a9ccbf2a274f85d4e/users/9081782f8f5d49bfa335028494240f23/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:58.930929 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.946212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1327/2635] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:58 2022] PUT /identity/v3/projects/d0024519726d4bc5a73c996909d68604/users/8670d0b5debd4fa49ebdf566c2b4b7d4/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:58.946610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6883f78ae4c94b6a9ccbf2a274f85d4e, user_id=9081782f8f5d49bfa335028494240f23, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:58.949485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:58.950266 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.971316 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1309/2636] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:58 2022] PUT /identity/v3/projects/6883f78ae4c94b6a9ccbf2a274f85d4e/users/9081782f8f5d49bfa335028494240f23/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:58.975214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:58.983633 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:59.025586 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:59.027470 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2fce6-b627-4de1-9ce3-84514b7257ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.063481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da865a-d2f2-49fe-9679-65947b297330 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.068979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.069829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.072085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.072531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.073453 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.075249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.091874 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1310/2637] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:59.096222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:59.107510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.108231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.108447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fc283-596c-44ab-8270-c41503597a9e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.108651 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.109018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.110018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.111796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.114076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae89a9-d47d-46e1-ad5a-79dd52101116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.114819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae89a9-d47d-46e1-ad5a-79dd52101116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.115170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae89a9-d47d-46e1-ad5a-79dd52101116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.115517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae89a9-d47d-46e1-ad5a-79dd52101116 admin admin] PATH_INFO: `/v3/projects/6883f78ae4c94b6a9ccbf2a274f85d4e/users/9081782f8f5d49bfa335028494240f23/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.117729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae89a9-d47d-46e1-ad5a-79dd52101116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6883f78ae4c94b6a9ccbf2a274f85d4e, user_id=9081782f8f5d49bfa335028494240f23, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.120711 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae89a9-d47d-46e1-ad5a-79dd52101116 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.128147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1328/2638] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:08:59.129900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:59.140075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1311/2639] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:59 2022] PUT /identity/v3/projects/6883f78ae4c94b6a9ccbf2a274f85d4e/users/9081782f8f5d49bfa335028494240f23/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:08:59.145624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157068e-bbf8-4a76-b972-d183fe369de5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.152290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.152838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.153350 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.188841 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] MFA Rules not processed for user `9081782f8f5d49bfa335028494240f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:59.190341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:59.209433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679db5c-fe1b-4baf-981e-089ec2d9a48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.210191 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679db5c-fe1b-4baf-981e-089ec2d9a48a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.210539 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679db5c-fe1b-4baf-981e-089ec2d9a48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.210886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679db5c-fe1b-4baf-981e-089ec2d9a48a admin admin] PATH_INFO: `/v3/projects/d0024519726d4bc5a73c996909d68604/users/8670d0b5debd4fa49ebdf566c2b4b7d4/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.213576 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679db5c-fe1b-4baf-981e-089ec2d9a48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0024519726d4bc5a73c996909d68604, user_id=8670d0b5debd4fa49ebdf566c2b4b7d4, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.216331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679db5c-fe1b-4baf-981e-089ec2d9a48a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.229907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1329/2640] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:08:59 2022] PUT /identity/v3/projects/d0024519726d4bc5a73c996909d68604/users/8670d0b5debd4fa49ebdf566c2b4b7d4/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:08:59.241675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.241995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.242274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.273724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] MFA Rules not processed for user `8670d0b5debd4fa49ebdf566c2b4b7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:08:59.275825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:59.277611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1312/2641] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:08:59.294725 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:59.296539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:59.339049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:59.360326 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5872ec1-8bce-4460-a24d-b74a72c9ed82 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.370497 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1330/2642] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:08:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:08:59.381550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:59.383947 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:08:59.422686 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.423400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.423806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.424076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.436554 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:08:59.449551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33637dd-f43a-46dd-9293-f4526fc29b5c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.453523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.454238 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.454514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.454817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.465333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.466889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.484079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1331/2643] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:08:59.507043 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.508515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.519448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1313/2644] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:08:59.563140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:08:59.572582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69806-9d9b-416a-a697-f47ebf84dc87 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:08:59.576603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:08:59.577213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:08:59.577494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:08:59.577765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:08:59.644905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:08:59.646350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:08:59.657052 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1332/2645] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:08:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:00.039828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:00.049751 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f07b3-a280-4022-bdf4-a16fea783a66 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:00.053628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b0338-03c9-4dad-9680-1666e2db29f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:00.054233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b0338-03c9-4dad-9680-1666e2db29f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:00.054504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b0338-03c9-4dad-9680-1666e2db29f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:00.054891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b0338-03c9-4dad-9680-1666e2db29f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:00.115354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b0338-03c9-4dad-9680-1666e2db29f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:00.116808 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b0338-03c9-4dad-9680-1666e2db29f0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:00.126213 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1314/2646] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:02.594780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:02.658126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1c258-0e02-4299-afa4-fdde8c707365 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:02.662699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:02.663326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:02.663613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:02.663906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] PATH_INFO: `/v3/users/250596ccca06425d90511eadd260acf5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:02.665021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=250596ccca06425d90511eadd260acf5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:02.667781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:02.696471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:02.714771 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250596ccca06425d90511eadd260acf5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:02.722217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250596ccca06425d90511eadd260acf5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:02.730011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250596ccca06425d90511eadd260acf5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:02.730635 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250596ccca06425d90511eadd260acf5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:02.742459 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250596ccca06425d90511eadd260acf5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:02.749002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1333/2647] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:02 2022] DELETE /identity/v3/users/250596ccca06425d90511eadd260acf5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:02.753830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:02.754282 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:02.754379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:02.773300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:02.774553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:02.870658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1315/2648] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:02.883614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:02.883841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:02.951388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238dd8a-8fd2-4253-88f6-b22d7b25e13f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:02.955046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:02.955679 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:02.955958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:02.956237 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:02.969394 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:02.970838 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:02.982252 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1334/2649] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:03.429173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:03.431166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:03.446925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888c23e-ecca-488d-8d25-d3f664bc03c2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:03.452201 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:03.452946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:03.453295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:03.453655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] PATH_INFO: `/v3/projects/061b02a6c1474f0283b260178f849576` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:03.454851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061b02a6c1474f0283b260178f849576)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:03.456891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:03.515576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:03.523512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061b02a6c1474f0283b260178f849576', 'request_id': 'req-bd7e0b63-a812-4a8f-8f68-03f1e541e214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.529149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061b02a6c1474f0283b260178f849576', 'request_id': 'req-bd7e0b63-a812-4a8f-8f68-03f1e541e214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.535911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061b02a6c1474f0283b260178f849576', 'request_id': 'req-bd7e0b63-a812-4a8f-8f68-03f1e541e214'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.549596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] The token cache is being invalidate because project 061b02a6c1474f0283b260178f849576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:03.550135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.550908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.552444 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1316/2650] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:03 2022] DELETE /identity/v3/projects/061b02a6c1474f0283b260178f849576 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:03.557232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:03.567335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb2375-cdf5-45e5-aa3b-7eea0e5d23a1 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:03.629742 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:03.630429 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:03.630648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:03.630922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] PATH_INFO: `/v3/users/feefd199c59f461c8a97bad2f3bdb086` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:03.632100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=feefd199c59f461c8a97bad2f3bdb086)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:03.634007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:03.688678 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feefd199c59f461c8a97bad2f3bdb086'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.691452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:03.697592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feefd199c59f461c8a97bad2f3bdb086'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.706323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feefd199c59f461c8a97bad2f3bdb086'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.707044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feefd199c59f461c8a97bad2f3bdb086'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.723695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feefd199c59f461c8a97bad2f3bdb086'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:03.730335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1335/2651] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:03 2022] DELETE /identity/v3/users/feefd199c59f461c8a97bad2f3bdb086 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:04.007462 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:04.008956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.024668 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e0b63-a812-4a8f-8f68-03f1e541e214 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:04.083095 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:04.083695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.083980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.084256 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] PATH_INFO: `/v3/projects/34143bec062b4a2aa91195b41ceebd2c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.085371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34143bec062b4a2aa91195b41ceebd2c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:04.087309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:04.134102 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34143bec062b4a2aa91195b41ceebd2c', 'request_id': 'req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.139483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34143bec062b4a2aa91195b41ceebd2c', 'request_id': 'req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.145685 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34143bec062b4a2aa91195b41ceebd2c', 'request_id': 'req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.159783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] The token cache is being invalidate because project 34143bec062b4a2aa91195b41ceebd2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:04.160075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.160907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.162604 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1317/2652] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:04 2022] DELETE /identity/v3/projects/34143bec062b4a2aa91195b41ceebd2c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:04.167629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:04.169037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.182335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1993a-2461-40c1-a98c-bbe461a4d61e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:04.248000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:04.248643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.248918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.249188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] PATH_INFO: `/v3/users/9081782f8f5d49bfa335028494240f23` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.250305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9081782f8f5d49bfa335028494240f23)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:04.252523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:04.307546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9081782f8f5d49bfa335028494240f23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.315803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9081782f8f5d49bfa335028494240f23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.323533 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9081782f8f5d49bfa335028494240f23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.324322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9081782f8f5d49bfa335028494240f23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.336326 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9081782f8f5d49bfa335028494240f23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.342961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1336/2653] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:04 2022] DELETE /identity/v3/users/9081782f8f5d49bfa335028494240f23 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:04.366982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:04.368587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.429176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836e2ef-a956-4cc0-ad7e-e14edeaa6450 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:04.440921 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:04.482406 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:04.483145 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.483439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.483744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.485490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.551055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:04.552691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c84175-f7b6-494e-b923-564579b24b3d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:04.562072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1318/2654] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:04.630992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:04.632588 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.698364 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:04.716908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a8d2-b655-4878-b29f-2c202469f970 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:04.721504 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:04.722137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.722417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.722699 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] PATH_INFO: `/v3/projects/6883f78ae4c94b6a9ccbf2a274f85d4e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.723872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6883f78ae4c94b6a9ccbf2a274f85d4e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:04.725769 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:04.781004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6883f78ae4c94b6a9ccbf2a274f85d4e', 'request_id': 'req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.789801 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6883f78ae4c94b6a9ccbf2a274f85d4e', 'request_id': 'req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.802020 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6883f78ae4c94b6a9ccbf2a274f85d4e', 'request_id': 'req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.814336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] The token cache is being invalidate because project 6883f78ae4c94b6a9ccbf2a274f85d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:04.814482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.815406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1b57f15-d3ce-45b1-99d9-14bfdae8021a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:04.816885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1337/2655] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:04 2022] DELETE /identity/v3/projects/6883f78ae4c94b6a9ccbf2a274f85d4e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:04.824955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.825394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.825745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.853858 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:04.855093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.929111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1319/2656] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:04.937963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.938283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.938659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.964241 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:04.965571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:04.974776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1338/2657] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:04.979167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:04.990341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4dd58-d4a6-41a6-9fdd-31841d4a618e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:04.996536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:04.997229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:04.997583 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:04.997977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:04.998707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:04.999088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.000840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.014984 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1320/2658] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:09:05.018417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:05.029915 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d17ca2-e7ca-49d6-8729-73e73b05a32b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:05.034621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:05.035216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.035491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.035911 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.036527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.038373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.069546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1339/2659] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:05 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:05.070594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:05.082463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbbff7-dd8a-449a-bb16-a702bb278670 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:05.135849 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:05.142563 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:05.143271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.143498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.143817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.144458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.146420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.169021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1321/2660] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:05 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:05.173413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:05.184896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62620b-cdad-4f97-9d10-3407e315c7d6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:05.190062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:05.190812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.191119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.191618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.192193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.193843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.205005 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1340/2661] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:05.211694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:05.222569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d18e7-181a-4cc0-a906-94887d128ae1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:05.227505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f1d54-867c-4113-a765-d20bd016c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:05.228915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1d54-867c-4113-a765-d20bd016c968 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.229231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1d54-867c-4113-a765-d20bd016c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.229505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1d54-867c-4113-a765-d20bd016c968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.230152 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1d54-867c-4113-a765-d20bd016c968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.231782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1d54-867c-4113-a765-d20bd016c968 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.244237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1322/2662] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:05.248606 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:05.259731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e366c-93b2-4a7b-aa12-704b4a2245fa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:05.264589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:05.265358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.265644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.265975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] PATH_INFO: `/v3/projects/a7c7bc5aff99400e9128ec5b6133f6b1/users/174a27114ea54630b82e09a566ee99fd/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.283056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c7bc5aff99400e9128ec5b6133f6b1, user_id=174a27114ea54630b82e09a566ee99fd, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.285793 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.308190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1341/2663] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:05 2022] PUT /identity/v3/projects/a7c7bc5aff99400e9128ec5b6133f6b1/users/174a27114ea54630b82e09a566ee99fd/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:05.319284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.319573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.319873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.347657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] MFA Rules not processed for user `174a27114ea54630b82e09a566ee99fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:05.348865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:05.416341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1323/2664] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:05.428178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:05.430163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:05.491836 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ff00-4af6-4926-8632-884686613541 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:05.495426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:05.496063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:05.496339 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:05.496610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:05.505454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:05.506882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:05.516593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1342/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:05 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:06.109657 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:06.111133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:06.168675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482b13b-ddc6-4d94-895b-ee03960fc2be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:06.172338 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:06.172969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:06.173242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:06.173559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:06.175271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:06.238696 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:06.240610 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cbf9f-a792-493b-857d-415d71158295 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:06.249887 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1324/2666] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:06.723034 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:06.724623 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:06.739088 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abab987-a49c-43d3-8ebc-398cbabf8276 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:06.743665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:06.744509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:06.744776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:06.745049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] PATH_INFO: `/v3/users/9902bfe1b82747c5be96717ad9bceffe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:06.746190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9902bfe1b82747c5be96717ad9bceffe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:06.748113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:06.758815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:06.800124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9902bfe1b82747c5be96717ad9bceffe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:06.807748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9902bfe1b82747c5be96717ad9bceffe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:06.814768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9902bfe1b82747c5be96717ad9bceffe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:06.815412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9902bfe1b82747c5be96717ad9bceffe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:06.828119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9902bfe1b82747c5be96717ad9bceffe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:06.834033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1343/2667] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:06 2022] DELETE /identity/v3/users/9902bfe1b82747c5be96717ad9bceffe => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:06.839249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:06.839540 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:06.839852 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:06.861775 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:06.863032 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:06.873262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:06.926721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1325/2668] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:06.937251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:06.938715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:07.009661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c69ad-57b2-4c5c-8f70-01ea8078dcb6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:07.014551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:07.015244 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:07.015622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:07.015967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:07.026543 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:07.028203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:07.038172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1344/2669] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:07.381791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:07.383401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:07.402955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01169d-ad58-4d9e-b3f5-fa410136242b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:07.412200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:07.412943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:07.413223 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:07.413504 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] PATH_INFO: `/v3/projects/bb64db740d0b493db58e67344d740823` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:07.414667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb64db740d0b493db58e67344d740823)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:07.416712 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:07.467125 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb64db740d0b493db58e67344d740823', 'request_id': 'req-d04d13d6-f355-4beb-aa15-18256bcb1c12'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.472810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb64db740d0b493db58e67344d740823', 'request_id': 'req-d04d13d6-f355-4beb-aa15-18256bcb1c12'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.479983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb64db740d0b493db58e67344d740823', 'request_id': 'req-d04d13d6-f355-4beb-aa15-18256bcb1c12'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.495631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] The token cache is being invalidate because project bb64db740d0b493db58e67344d740823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:07.495940 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.496606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.498124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1326/2670] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:07 2022] DELETE /identity/v3/projects/bb64db740d0b493db58e67344d740823 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:07.502600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:07.513399 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279910-54b4-4534-821e-1729ca9cebe3 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:07.576040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:07.576608 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:07.576894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:07.577172 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] PATH_INFO: `/v3/users/f19925dbf5ed441a886c8f4693bbca10` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:07.578386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19925dbf5ed441a886c8f4693bbca10)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:07.580420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:07.619830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:07.634729 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19925dbf5ed441a886c8f4693bbca10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.644012 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19925dbf5ed441a886c8f4693bbca10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.652296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19925dbf5ed441a886c8f4693bbca10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.653123 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19925dbf5ed441a886c8f4693bbca10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.667807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19925dbf5ed441a886c8f4693bbca10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:07.674214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1345/2671] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:07 2022] DELETE /identity/v3/users/f19925dbf5ed441a886c8f4693bbca10 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:07.956176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:07.957990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:07.973093 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d13d6-f355-4beb-aa15-18256bcb1c12 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:08.052151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:08.052934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:08.053226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:08.053529 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] PATH_INFO: `/v3/projects/fcb0c96e7ac9484c9b7dc5602166630f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:08.054652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb0c96e7ac9484c9b7dc5602166630f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:08.057626 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:08.107422 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:08.114189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb0c96e7ac9484c9b7dc5602166630f', 'request_id': 'req-77731c7f-08e5-4672-a2f2-1821fe7ffafb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.122218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb0c96e7ac9484c9b7dc5602166630f', 'request_id': 'req-77731c7f-08e5-4672-a2f2-1821fe7ffafb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.131298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb0c96e7ac9484c9b7dc5602166630f', 'request_id': 'req-77731c7f-08e5-4672-a2f2-1821fe7ffafb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.147324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] The token cache is being invalidate because project fcb0c96e7ac9484c9b7dc5602166630f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:08.148058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.148298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.150576 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1327/2672] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:07 2022] DELETE /identity/v3/projects/fcb0c96e7ac9484c9b7dc5602166630f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:08.154643 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:08.157212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:08.174304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386607a-03d8-4f9f-b654-b219219dc0da admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:08.262782 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:08.263353 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:08.264374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:08.264374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] PATH_INFO: `/v3/users/8670d0b5debd4fa49ebdf566c2b4b7d4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:08.265089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8670d0b5debd4fa49ebdf566c2b4b7d4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:08.268932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:08.331202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8670d0b5debd4fa49ebdf566c2b4b7d4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.339483 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8670d0b5debd4fa49ebdf566c2b4b7d4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.348500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8670d0b5debd4fa49ebdf566c2b4b7d4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.349225 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8670d0b5debd4fa49ebdf566c2b4b7d4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.361637 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26277a55-ae52-4691-8cee-9efb53fbad47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8670d0b5debd4fa49ebdf566c2b4b7d4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.368044 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1346/2673] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:08 2022] DELETE /identity/v3/users/8670d0b5debd4fa49ebdf566c2b4b7d4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:08.668963 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:08.670979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:08.686762 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77731c7f-08e5-4672-a2f2-1821fe7ffafb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:08.764877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:08.765521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:08.765804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:08.766084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] PATH_INFO: `/v3/projects/d0024519726d4bc5a73c996909d68604` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:08.767351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0024519726d4bc5a73c996909d68604)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:08.769351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:08.817954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0024519726d4bc5a73c996909d68604', 'request_id': 'req-879ac2fc-bd0f-4593-aea7-48e1ef117077'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.824645 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0024519726d4bc5a73c996909d68604', 'request_id': 'req-879ac2fc-bd0f-4593-aea7-48e1ef117077'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.831014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0024519726d4bc5a73c996909d68604', 'request_id': 'req-879ac2fc-bd0f-4593-aea7-48e1ef117077'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.849646 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] The token cache is being invalidate because project d0024519726d4bc5a73c996909d68604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:08.849944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.850594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-879ac2fc-bd0f-4593-aea7-48e1ef117077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:08.852137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1328/2674] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:08 2022] DELETE /identity/v3/projects/d0024519726d4bc5a73c996909d68604 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:08.859124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:08.859279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:08.859454 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:08.880494 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:08.881715 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:08.906464 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:08.944644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1347/2675] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:08.954783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:08.955070 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:08.955348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:08.974983 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:08.976200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:08.984566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1329/2676] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:08.988689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:08.999002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689bb9f-ce91-47e9-a6e6-84414015ed77 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.012564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.013279 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.013753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.014113 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.015063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:09.015673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.017295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.032932 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1348/2677] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:09:09.037528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.050448 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e12d7-404c-47a1-883c-7ad588566e7d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.055498 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.056060 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.056346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.056624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.057721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.060211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.102440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1330/2678] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:09 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:09.107043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.121705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e0316-84b6-49bd-8f02-8ed8cb1815ae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.210208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.210822 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.211083 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.211350 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.212115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.214803 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.236710 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1349/2679] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:09 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:09.240948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.253832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdf5a7-2ee5-41f2-9f37-7226d0fb9e6d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.261021 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.261587 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.261873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.262153 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.262802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.264544 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.279457 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1331/2680] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:09.280783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.296622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ec9e8-2631-42d5-8d40-33354f809ba3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.301228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a598d-59ef-453e-8303-56a24ed37149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.301826 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a598d-59ef-453e-8303-56a24ed37149 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.302105 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a598d-59ef-453e-8303-56a24ed37149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.302376 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a598d-59ef-453e-8303-56a24ed37149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.303047 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a598d-59ef-453e-8303-56a24ed37149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.304808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a598d-59ef-453e-8303-56a24ed37149 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.316179 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1350/2681] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:09.320030 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.331045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cfba-7ab7-4adc-9528-0daeb3642720 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.335851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.336632 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.336914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.337827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] PATH_INFO: `/v3/projects/25a126ef39c84b09bfda9786d6a67020/users/4294b51f30054c2d898d6305f0c1548d/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.353536 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a126ef39c84b09bfda9786d6a67020, user_id=4294b51f30054c2d898d6305f0c1548d, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.356276 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.359398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:09.372781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1332/2682] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:09 2022] PUT /identity/v3/projects/25a126ef39c84b09bfda9786d6a67020/users/4294b51f30054c2d898d6305f0c1548d/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:09.384723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.385090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.385552 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.415959 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] MFA Rules not processed for user `4294b51f30054c2d898d6305f0c1548d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:09.417288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:09.482990 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1351/2683] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:09.487923 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.574595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1977523-f2b5-4298-ab02-ab811e86b28e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.579384 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.580570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.580570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.580570 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.581203 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.583089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.614127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1333/2684] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:09 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:09:09.619018 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.635300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600285c4-3347-4d84-b539-1a1d2521584b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.661804 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.664527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:09.667960 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:09.711141 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.711794 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.712137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.712416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.713061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.714969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.734619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc58de-dd20-4d74-bd3c-e10dadb24f6d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.737795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1352/2685] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:09 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:09.739514 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.740195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.740494 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.740761 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] PATH_INFO: `/v3/users/0bb72c5f52fc498aa60eecba3bd5248a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.741932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb72c5f52fc498aa60eecba3bd5248a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.742057 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.744518 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.753247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bffbc-3f68-4493-af4a-9c89fb70bae1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.758159 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.758890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.759184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.759455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.760152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.761774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.775717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1353/2686] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:09.778718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.791901 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a57a7-0aa5-42d6-89da-b5a1e1504496 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.797413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.798080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.798362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.798636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] PATH_INFO: `/v3/projects/2029afa4448d49c79b8c8f84e96e11ca/users/0b712c0a3dc24c5f85e4217daa1c9295/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.801254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb72c5f52fc498aa60eecba3bd5248a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:09.810501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb72c5f52fc498aa60eecba3bd5248a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:09.811270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb72c5f52fc498aa60eecba3bd5248a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:09.821464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb72c5f52fc498aa60eecba3bd5248a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:09.824272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2029afa4448d49c79b8c8f84e96e11ca, user_id=0b712c0a3dc24c5f85e4217daa1c9295, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.826943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:09.842300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb72c5f52fc498aa60eecba3bd5248a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:09.846277 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1354/2687] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:09 2022] PUT /identity/v3/projects/2029afa4448d49c79b8c8f84e96e11ca/users/0b712c0a3dc24c5f85e4217daa1c9295/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:09.850653 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:09.851429 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:09.853310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:09.856386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1334/2688] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:09 2022] DELETE /identity/v3/users/0bb72c5f52fc498aa60eecba3bd5248a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:09.938499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72f7b1-d079-49b9-97db-4390f9aeb2b1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:09.996137 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:09.996751 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:09.997033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:09.997308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:09.997957 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:09.999544 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.016026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1355/2689] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:10.020690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.031477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d06eb9-4e12-4b85-bac9-ae94e69f05e1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.037241 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e3f37-b586-4219-a69c-6c458fa67f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.037918 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e3f37-b586-4219-a69c-6c458fa67f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.038200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e3f37-b586-4219-a69c-6c458fa67f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.038471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e3f37-b586-4219-a69c-6c458fa67f48 admin admin] PATH_INFO: `/v3/projects/2029afa4448d49c79b8c8f84e96e11ca/users/0b712c0a3dc24c5f85e4217daa1c9295/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.040513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e3f37-b586-4219-a69c-6c458fa67f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2029afa4448d49c79b8c8f84e96e11ca, user_id=0b712c0a3dc24c5f85e4217daa1c9295, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.043307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e3f37-b586-4219-a69c-6c458fa67f48 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.046347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.048289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.056335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1335/2690] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:10 2022] PUT /identity/v3/projects/2029afa4448d49c79b8c8f84e96e11ca/users/0b712c0a3dc24c5f85e4217daa1c9295/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:10.065255 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f3b59-738e-4838-aa93-b7d22b6e0d11 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.067147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.067509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.067894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.093119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:10.100343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] MFA Rules not processed for user `0b712c0a3dc24c5f85e4217daa1c9295`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:10.101881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.140000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.140940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.140940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.141201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] PATH_INFO: `/v3/projects/10c7e20a9669439fa43853dab6f9851c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.143071 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c7e20a9669439fa43853dab6f9851c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.145114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.180300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1336/2691] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:10.193464 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.195101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.195833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c7e20a9669439fa43853dab6f9851c', 'request_id': 'req-3104a04d-be9c-4408-b21a-bac5d7de3f4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.204474 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c7e20a9669439fa43853dab6f9851c', 'request_id': 'req-3104a04d-be9c-4408-b21a-bac5d7de3f4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.211383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c7e20a9669439fa43853dab6f9851c', 'request_id': 'req-3104a04d-be9c-4408-b21a-bac5d7de3f4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.222704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] The token cache is being invalidate because project 10c7e20a9669439fa43853dab6f9851c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:10.222985 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.223670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3104a04d-be9c-4408-b21a-bac5d7de3f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.225147 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1356/2692] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:10 2022] DELETE /identity/v3/projects/10c7e20a9669439fa43853dab6f9851c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:10.229936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281740c-3d16-495b-acec-633fdee56e60 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.230213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281740c-3d16-495b-acec-633fdee56e60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.230477 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281740c-3d16-495b-acec-633fdee56e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.251223 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0281740c-3d16-495b-acec-633fdee56e60 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:10.252798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0281740c-3d16-495b-acec-633fdee56e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.259479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a20bd-ba0a-4d6e-ba65-ae6ae39ed3fa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.333583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1357/2693] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:10.337363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.337923 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.338978 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.339650 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.340780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.341205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.341624 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.343930 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.363231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:10.364595 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.375304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1358/2694] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:10.379958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.390402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b36a9-00d8-4262-a8dd-87d2bbf5016d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.395724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.396395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.396750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.397091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.397876 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:10.398247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.400232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.409934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:10.422129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1359/2695] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:09:10.423148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.434780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60066-e3b3-442c-88a0-46eda1a85c98 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.440023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.440174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.440629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.440909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.441185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.441782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.441897 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.443767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.452644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1337/2696] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:09:10.485815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1360/2697] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:10.490014 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.507439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1efcdc-ab46-4595-9291-2f11bd3f4479 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.571713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.572556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.572838 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.573144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.573830 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.576298 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.599166 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1338/2698] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:10.604546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.612783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.615508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.617383 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fa11b-bd72-49e0-9167-2c0fa50c86a3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.624058 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.624748 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.625038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.625315 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.625967 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.628038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.631943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c6f02-45d7-4fdd-a34c-e2271b6ceb75 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.633603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:10.638268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.639167 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.639303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1361/2699] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:10.641758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.641988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] PATH_INFO: `/v3/users/4294b51f30054c2d898d6305f0c1548d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.643707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4294b51f30054c2d898d6305f0c1548d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.643980 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.646749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.655214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166d637-324c-4001-949f-d2e88f613575 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.660006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.660604 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.660885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.661158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.661799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.663367 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.674351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1362/2700] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:10.678708 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.690151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccbdbd-6426-4fdd-9db4-68b0f182b92e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.695945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16262506-071e-4737-bdde-ea76708da618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.697166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262506-071e-4737-bdde-ea76708da618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.697445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262506-071e-4737-bdde-ea76708da618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.697722 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262506-071e-4737-bdde-ea76708da618 admin admin] PATH_INFO: `/v3/projects/b7a4467f77bb4b2bbe6aa43994740ca7/users/b11fb26413a946c8b58530c3d33b1686/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.699097 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4294b51f30054c2d898d6305f0c1548d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.706133 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4294b51f30054c2d898d6305f0c1548d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.706842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4294b51f30054c2d898d6305f0c1548d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.714393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16262506-071e-4737-bdde-ea76708da618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a4467f77bb4b2bbe6aa43994740ca7, user_id=b11fb26413a946c8b58530c3d33b1686, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.715206 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4294b51f30054c2d898d6305f0c1548d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.717375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16262506-071e-4737-bdde-ea76708da618 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.728701 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ce3d9f0d-bc4e-4413-8fc1-336efc67d93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4294b51f30054c2d898d6305f0c1548d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:10.731387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1363/2701] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:10 2022] PUT /identity/v3/projects/b7a4467f77bb4b2bbe6aa43994740ca7/users/b11fb26413a946c8b58530c3d33b1686/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:10.735844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1339/2702] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:10 2022] DELETE /identity/v3/users/4294b51f30054c2d898d6305f0c1548d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:10.740235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.740624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.740903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.741957 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.742244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.742519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.760880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:10.762236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.770439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] MFA Rules not processed for user `b11fb26413a946c8b58530c3d33b1686`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:10.771800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.804259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:10.827510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1364/2703] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:10.833868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.835226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.842972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1340/2704] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:10.847430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.848851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:10.863452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa70f1-8813-4986-8b21-bf3f69e05955 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.868532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.869137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.869413 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.869678 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.870293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.872239 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.897016 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb6e8-6410-407e-86eb-643e07697860 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.904905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1341/2705] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:10.905053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.905053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.905053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.905053 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.905901 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:10.913227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.914636 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:10.925265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1365/2706] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:09:10.969870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394a278-9697-4518-aa08-7df8b0ca915a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:10.974915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:10.975541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:10.976002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:10.976290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:10.976942 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:10.979487 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.000354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1342/2707] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:10 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:11.004882 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.015065 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1505c-15a8-47b4-bc06-6a35d80c269a service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.019835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.020547 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.020720 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.021120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.021621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.023305 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.035424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1366/2708] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:11.039638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.050370 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002deb8f-ad93-424f-be9b-c8582548c411 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.055258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.055969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.056279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.056524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] PATH_INFO: `/v3/projects/ab73f26ef81a4324a7052f4c54d99e4b/users/6cff75643ccf431984bf09fa27d4806a/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.075633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab73f26ef81a4324a7052f4c54d99e4b, user_id=6cff75643ccf431984bf09fa27d4806a, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.078410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.095507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1343/2709] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:11 2022] PUT /identity/v3/projects/ab73f26ef81a4324a7052f4c54d99e4b/users/6cff75643ccf431984bf09fa27d4806a/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:11.099389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.111235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb444844-9596-4a3c-b114-1619f73b242c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.163303 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:11.170622 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.171645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.171927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.172194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.172837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.174420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.184753 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1367/2710] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:11.189046 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.202832 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28bf27-8a72-48b2-9ae2-1d70bddcfd6a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.208079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25c4b6-b443-4059-ba30-82aa7b2606dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.208743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25c4b6-b443-4059-ba30-82aa7b2606dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.209025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25c4b6-b443-4059-ba30-82aa7b2606dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.209307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25c4b6-b443-4059-ba30-82aa7b2606dc admin admin] PATH_INFO: `/v3/projects/ab73f26ef81a4324a7052f4c54d99e4b/users/6cff75643ccf431984bf09fa27d4806a/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.211215 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25c4b6-b443-4059-ba30-82aa7b2606dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab73f26ef81a4324a7052f4c54d99e4b, user_id=6cff75643ccf431984bf09fa27d4806a, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.215373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25c4b6-b443-4059-ba30-82aa7b2606dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.216628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.218992 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.226426 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1344/2711] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:11 2022] PUT /identity/v3/projects/ab73f26ef81a4324a7052f4c54d99e4b/users/6cff75643ccf431984bf09fa27d4806a/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:11.234549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1a784-f84f-413f-8f42-86997a20e23e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.242948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.246913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.247293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.278404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] MFA Rules not processed for user `6cff75643ccf431984bf09fa27d4806a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:11.279644 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.299021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.299695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.299976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.300250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] PATH_INFO: `/v3/projects/25a126ef39c84b09bfda9786d6a67020` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.301387 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a126ef39c84b09bfda9786d6a67020)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.303289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.326144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:11.350375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a126ef39c84b09bfda9786d6a67020', 'request_id': 'req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.351135 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1345/2712] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:11.362416 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a126ef39c84b09bfda9786d6a67020', 'request_id': 'req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.362836 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.364272 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.369377 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a126ef39c84b09bfda9786d6a67020', 'request_id': 'req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.381444 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] The token cache is being invalidate because project 25a126ef39c84b09bfda9786d6a67020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:11.381848 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.382614 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.384767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1368/2713] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:11 2022] DELETE /identity/v3/projects/25a126ef39c84b09bfda9786d6a67020 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:11.388694 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.390035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.426179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733fa39-1cee-423d-aecd-4801ac8ff283 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.453831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a699a3-e5e9-4bb6-9a23-5ae2b4a9eab2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.458482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.459108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.459393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.459697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] PATH_INFO: `/v3/users/0b712c0a3dc24c5f85e4217daa1c9295` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.460830 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b712c0a3dc24c5f85e4217daa1c9295)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.462837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.477170 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.477823 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.478173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.478468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.480440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.489808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:11.512551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b712c0a3dc24c5f85e4217daa1c9295'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.522165 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b712c0a3dc24c5f85e4217daa1c9295'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.529920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b712c0a3dc24c5f85e4217daa1c9295'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.530674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b712c0a3dc24c5f85e4217daa1c9295'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.545719 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b712c0a3dc24c5f85e4217daa1c9295'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.551159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.552627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.555563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1369/2714] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:11 2022] DELETE /identity/v3/users/0b712c0a3dc24c5f85e4217daa1c9295 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:11.564417 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1346/2715] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:09:11.686144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.687666 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.755257 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed45b91-3aff-4c63-b21a-23e6b01c63b8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.760325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.761114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.761589 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.761919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.763971 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:11.834332 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.836263 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5cd7f-0762-46fb-b9ef-168f08bfccb7 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.850700 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1370/2716] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:11.856815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:11.871297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738780e-28ac-4bd0-8105-1c85d1569897 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:11.923077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:11.934313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:11.934802 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:11.935084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:11.935358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] PATH_INFO: `/v3/projects/2029afa4448d49c79b8c8f84e96e11ca` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:11.936542 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2029afa4448d49c79b8c8f84e96e11ca)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:11.938447 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:11.989143 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2029afa4448d49c79b8c8f84e96e11ca', 'request_id': 'req-8ce15244-ff19-4776-9cff-2eaec6cfd48d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:11.996271 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2029afa4448d49c79b8c8f84e96e11ca', 'request_id': 'req-8ce15244-ff19-4776-9cff-2eaec6cfd48d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.003358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2029afa4448d49c79b8c8f84e96e11ca', 'request_id': 'req-8ce15244-ff19-4776-9cff-2eaec6cfd48d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.016801 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] The token cache is being invalidate because project 2029afa4448d49c79b8c8f84e96e11ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:12.017098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.017791 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8ce15244-ff19-4776-9cff-2eaec6cfd48d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.019627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1347/2717] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:11 2022] DELETE /identity/v3/projects/2029afa4448d49c79b8c8f84e96e11ca => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:12.026201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.026568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.026854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.047620 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:12.049178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:12.104927 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:12.123904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1371/2718] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:12.128805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.129098 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.129376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.149082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:12.150331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:12.158765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1348/2719] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:12.162974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.174980 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491b4af-3789-4490-b3f7-315fc2737feb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.180100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.181471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.181827 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.182154 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.182879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:12.183186 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.184979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.197046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1372/2720] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:09:12.201534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.210731 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe28c9-7596-43ed-9831-2f82d322b3ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.216567 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.217556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.217866 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.218134 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.218786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.221061 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.249667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1349/2721] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:12 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:12.254102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.263934 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c2071-8191-4f07-b16a-155e5151a528 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.319213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.319837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.320116 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.320386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.321002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.323110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.344508 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1373/2722] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:12 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:09:12.348717 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.360616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443888-7084-4e85-888f-147b8e38bfa7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.365328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.365929 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.366233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.366523 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.367163 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.368859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.379347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1350/2723] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:12.383379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.393450 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54e780-80cd-48d0-af9a-f0fff004bc35 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.398529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba92d8-39d1-41f1-8cb6-f41fc1062a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.399129 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba92d8-39d1-41f1-8cb6-f41fc1062a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.400051 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba92d8-39d1-41f1-8cb6-f41fc1062a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.400446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba92d8-39d1-41f1-8cb6-f41fc1062a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.401095 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba92d8-39d1-41f1-8cb6-f41fc1062a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.402690 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba92d8-39d1-41f1-8cb6-f41fc1062a2c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.413301 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1374/2724] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:12.418290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.428895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9435-810e-4cd4-ad42-789b19a2526d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.433530 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.434178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.434456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.434727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] PATH_INFO: `/v3/projects/7b3521a89c164be1be6dd934cfa5d40f/users/edef859df06b41bfb3bbc85067422f8f/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.450158 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3521a89c164be1be6dd934cfa5d40f, user_id=edef859df06b41bfb3bbc85067422f8f, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.452756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.466252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1351/2725] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:12 2022] PUT /identity/v3/projects/7b3521a89c164be1be6dd934cfa5d40f/users/edef859df06b41bfb3bbc85067422f8f/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:12.476640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.476930 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.477232 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.504768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] MFA Rules not processed for user `edef859df06b41bfb3bbc85067422f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:12.505961 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:12.572861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1375/2726] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:12.583970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.585655 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:12.682461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.683952 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:12.741717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:12.756442 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93a891-d3a6-4c39-975b-ff5e880be677 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.761724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.762370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.762681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.762958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] PATH_INFO: `/v3/users/174a27114ea54630b82e09a566ee99fd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.764127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=174a27114ea54630b82e09a566ee99fd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.766046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.772515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6fd84-5279-4a47-a8dd-ff55bbb582ef admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:12.776828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd9801-f083-4a2a-a3e9-e4cb79a4a90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:12.777723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9801-f083-4a2a-a3e9-e4cb79a4a90b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.777999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9801-f083-4a2a-a3e9-e4cb79a4a90b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.778264 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9801-f083-4a2a-a3e9-e4cb79a4a90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.787151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd9801-f083-4a2a-a3e9-e4cb79a4a90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:12.788627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd9801-f083-4a2a-a3e9-e4cb79a4a90b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:12.799199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1352/2727] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:12.814647 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174a27114ea54630b82e09a566ee99fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.821974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174a27114ea54630b82e09a566ee99fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.829340 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174a27114ea54630b82e09a566ee99fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.830258 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174a27114ea54630b82e09a566ee99fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.848581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174a27114ea54630b82e09a566ee99fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:12.855406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1376/2728] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:12 2022] DELETE /identity/v3/users/174a27114ea54630b82e09a566ee99fd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:12.859726 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:12.860243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:12.860243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:12.880565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:12.882151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:12.896399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:12.952064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1353/2729] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:12.962685 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:12.964166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:13.025327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00300a1a-bc50-4e79-bc7e-ba3a4d17c82a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.030115 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9199aa-40ec-4d96-808e-8db5c00315c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.030725 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9199aa-40ec-4d96-808e-8db5c00315c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.030998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9199aa-40ec-4d96-808e-8db5c00315c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.031325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9199aa-40ec-4d96-808e-8db5c00315c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.041563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9199aa-40ec-4d96-808e-8db5c00315c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.043027 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9199aa-40ec-4d96-808e-8db5c00315c5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.053010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1377/2730] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:13.353094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.354621 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:13.372704 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c63a3-4258-4855-b514-46b8c83d211f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.378382 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.379024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.379303 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.379575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] PATH_INFO: `/v3/projects/a7c7bc5aff99400e9128ec5b6133f6b1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.380837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c7bc5aff99400e9128ec5b6133f6b1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.382724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.440581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c7bc5aff99400e9128ec5b6133f6b1', 'request_id': 'req-90c9d0c9-09d7-4329-8879-5f2e328c785b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:13.448345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c7bc5aff99400e9128ec5b6133f6b1', 'request_id': 'req-90c9d0c9-09d7-4329-8879-5f2e328c785b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:13.456348 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c7bc5aff99400e9128ec5b6133f6b1', 'request_id': 'req-90c9d0c9-09d7-4329-8879-5f2e328c785b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:13.472199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] The token cache is being invalidate because project a7c7bc5aff99400e9128ec5b6133f6b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:13.472489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:13.473174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-90c9d0c9-09d7-4329-8879-5f2e328c785b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:13.475432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1354/2731] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:13 2022] DELETE /identity/v3/projects/a7c7bc5aff99400e9128ec5b6133f6b1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:13.481140 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.481599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.481713 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.500804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:13.502259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:13.561202 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1378/2732] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:09:13.570038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.570325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.570599 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.591744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:13.592946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:13.601267 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1355/2733] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:13.605995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.616440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4729f-9f42-47ad-a314-344c75a10f74 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.621158 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.621788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.622072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.622341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.622996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:13.623296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.624909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.637948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1379/2734] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:09:13.642593 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.653199 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4693-6b25-40e2-9fcb-c6a347dbec68 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.657965 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.658566 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.658846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.659119 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.659785 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.661703 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.688315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1356/2735] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:13 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:09:13.692506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.698630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:13.704839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bf2fb-3f3b-458d-bca7-a128eb469656 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.762192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.762852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.763136 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.763403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.764055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.765943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.787331 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1380/2736] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:13 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:09:13.791706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.802005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d094ef-1e2d-4a40-85e3-1e50cdab3160 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.807205 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.807824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.808104 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.808369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.808999 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.810606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.820263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1357/2737] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:13.825354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.835370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c58e-194a-4a72-954e-ab52add2a43b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.840278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7839be-6d7d-4161-a159-c61e27e9e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.840880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7839be-6d7d-4161-a159-c61e27e9e659 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.841162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7839be-6d7d-4161-a159-c61e27e9e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.841435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7839be-6d7d-4161-a159-c61e27e9e659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.842079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7839be-6d7d-4161-a159-c61e27e9e659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.843698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7839be-6d7d-4161-a159-c61e27e9e659 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.854581 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1381/2738] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:13.858652 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:13.869456 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e22d6-f5b3-4c13-a5d7-f5d28a9cbb29 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:13.874130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:13.874776 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.875058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.875333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] PATH_INFO: `/v3/projects/272a737decfd4dba9538dffbc7c1db5b/users/97ab9c24c5404e1892bb44b09be583fe/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.890734 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272a737decfd4dba9538dffbc7c1db5b, user_id=97ab9c24c5404e1892bb44b09be583fe, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:13.893614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:13.906809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1358/2739] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:13 2022] PUT /identity/v3/projects/272a737decfd4dba9538dffbc7c1db5b/users/97ab9c24c5404e1892bb44b09be583fe/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:13.911886 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:13.912180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:13.912457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:13.932007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:13.933226 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:13.991079 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1382/2740] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:14.001080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:14.003025 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:14.048640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:14.063885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63797517-3260-4ff3-985d-7e4d33213323 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:14.067515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c7b06-aaa1-45b5-a001-de65e32a6e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:14.068150 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7b06-aaa1-45b5-a001-de65e32a6e27 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:14.068430 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7b06-aaa1-45b5-a001-de65e32a6e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:14.068709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7b06-aaa1-45b5-a001-de65e32a6e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:14.078437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c7b06-aaa1-45b5-a001-de65e32a6e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:14.079842 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c7b06-aaa1-45b5-a001-de65e32a6e27 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:14.089185 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1359/2741] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:14.496041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:14.497554 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:14.509523 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ccad2-f558-4921-bc90-096a84e907a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:14.514333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:14.515042 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:14.515327 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:14.515704 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] PATH_INFO: `/v3/users/b11fb26413a946c8b58530c3d33b1686` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:14.516859 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11fb26413a946c8b58530c3d33b1686)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:14.518775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:14.568217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11fb26413a946c8b58530c3d33b1686'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:14.575692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11fb26413a946c8b58530c3d33b1686'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:14.584120 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11fb26413a946c8b58530c3d33b1686'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:14.584783 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11fb26413a946c8b58530c3d33b1686'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:14.595936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:14.598262 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11fb26413a946c8b58530c3d33b1686'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:14.607254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1383/2742] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:14 2022] DELETE /identity/v3/users/b11fb26413a946c8b58530c3d33b1686 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:14.611795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:14.612106 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:14.612386 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:14.631950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:14.633161 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:14.701721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1360/2743] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:14.711426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:14.713601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:14.786894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293752-3b04-4f0b-83fa-ec33b9720ad2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:14.792475 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:14.793100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:14.793379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:14.793656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:14.803328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:14.804802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:14.815847 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1384/2744] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:15.541594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:15.543137 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:15.557569 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fb8d-a997-4019-a37a-96e89534cd48 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:15.562502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:15.563148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:15.563560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:15.564051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] PATH_INFO: `/v3/projects/b7a4467f77bb4b2bbe6aa43994740ca7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:15.565074 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a4467f77bb4b2bbe6aa43994740ca7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:15.567047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:15.638252 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a4467f77bb4b2bbe6aa43994740ca7', 'request_id': 'req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.646148 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a4467f77bb4b2bbe6aa43994740ca7', 'request_id': 'req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.653476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a4467f77bb4b2bbe6aa43994740ca7', 'request_id': 'req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.669216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] The token cache is being invalidate because project b7a4467f77bb4b2bbe6aa43994740ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:15.669819 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.670164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.671913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1361/2745] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:15 2022] DELETE /identity/v3/projects/b7a4467f77bb4b2bbe6aa43994740ca7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:15.676231 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:15.687378 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca20a29-f900-4be2-ac62-fd1973cb9909 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:15.746951 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:15.747958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:15.748240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:15.748560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] PATH_INFO: `/v3/users/6cff75643ccf431984bf09fa27d4806a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:15.750556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cff75643ccf431984bf09fa27d4806a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:15.753482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:15.812215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cff75643ccf431984bf09fa27d4806a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.824724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cff75643ccf431984bf09fa27d4806a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.834875 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cff75643ccf431984bf09fa27d4806a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.836630 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cff75643ccf431984bf09fa27d4806a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.849275 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bf1994f-f018-4632-933d-1c2582bd9469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cff75643ccf431984bf09fa27d4806a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:15.856245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1385/2746] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:15 2022] DELETE /identity/v3/users/6cff75643ccf431984bf09fa27d4806a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:16.178449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.180365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:16.201371 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45971d-7fbc-484f-85e4-d9fa22b9cc1b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.280290 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.280893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.281663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.281663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] PATH_INFO: `/v3/projects/ab73f26ef81a4324a7052f4c54d99e4b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.282774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab73f26ef81a4324a7052f4c54d99e4b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:16.284834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:16.293989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:16.353313 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab73f26ef81a4324a7052f4c54d99e4b', 'request_id': 'req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:16.359270 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab73f26ef81a4324a7052f4c54d99e4b', 'request_id': 'req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:16.380341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab73f26ef81a4324a7052f4c54d99e4b', 'request_id': 'req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:16.395047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] The token cache is being invalidate because project ab73f26ef81a4324a7052f4c54d99e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:16.395433 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:16.396676 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9107e2ad-d329-4e6b-a99f-5e336b70bdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:16.398243 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1362/2747] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:16 2022] DELETE /identity/v3/projects/ab73f26ef81a4324a7052f4c54d99e4b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:16.404023 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.404314 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.404590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.425607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:16.427420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:16.443096 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:16.515775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1386/2748] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:16.526869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.527497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.527950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.551266 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:16.553345 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:16.563618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1363/2749] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:16.568030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.584480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f456927-23f8-40a9-85f9-d72414d3d324 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.593384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.593888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.594167 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.596966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.596966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:16.599355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:16.603868 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:16.622110 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1387/2750] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:09:16.626268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.641508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db11f8e-02c0-4ee0-b6dd-d784fd8cfdd5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.647794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.648699 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.649218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.651249 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.652317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:16.655737 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:16.713636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1364/2751] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:16 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:16.718126 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.735245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03c339-07d8-4ef3-9979-0b73e2a7b2ac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.841385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.842011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.842287 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.842558 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.843203 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:16.845890 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:16.887568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1388/2752] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:16 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:16.888230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.903966 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd80d6b-c56c-49a3-9867-0241818f6729 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.918764 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.919418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.919727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.920362 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.921502 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:16.923296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:16.938524 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.943805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1365/2753] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:16.949471 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7a9f-a4c4-40b6-b373-5ed405cf61ae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.954363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331c82f-b577-4e53-951b-2f8b3e09fd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.954969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331c82f-b577-4e53-951b-2f8b3e09fd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.955245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331c82f-b577-4e53-951b-2f8b3e09fd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.955515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331c82f-b577-4e53-951b-2f8b3e09fd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:16.956192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331c82f-b577-4e53-951b-2f8b3e09fd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:16.957780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331c82f-b577-4e53-951b-2f8b3e09fd9f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:16.970739 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1389/2754] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:16.974738 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:16.985969 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2442b72-697d-4546-815d-678a472a6fcb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:16.990880 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:16.991537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:16.991845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:16.992317 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] PATH_INFO: `/v3/projects/fcd635315d2e40bfb391108a5f4da431/users/8016241368a14156badd1f862f48d5bc/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:17.013173 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd635315d2e40bfb391108a5f4da431, user_id=8016241368a14156badd1f862f48d5bc, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:17.015934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:17.032247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1366/2755] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:16 2022] PUT /identity/v3/projects/fcd635315d2e40bfb391108a5f4da431/users/8016241368a14156badd1f862f48d5bc/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:17.042407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362cff9-c8c6-4c13-9fbb-f5f8c5e55d63 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:17.042750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362cff9-c8c6-4c13-9fbb-f5f8c5e55d63 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:17.043086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362cff9-c8c6-4c13-9fbb-f5f8c5e55d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:17.072723 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d362cff9-c8c6-4c13-9fbb-f5f8c5e55d63 None None] MFA Rules not processed for user `8016241368a14156badd1f862f48d5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:17.074035 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d362cff9-c8c6-4c13-9fbb-f5f8c5e55d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:17.151180 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1390/2756] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:17.164653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:17.167053 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:17.232829 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103272-83e6-403e-b394-688d47d70431 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:17.236729 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:17.237090 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:17.237375 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:17.237653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:17.247368 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:17.248907 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:17.260130 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1367/2757] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:18.959759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:18.960254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:18.960355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:18.990438 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:19.020658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] MFA Rules not processed for user `97ab9c24c5404e1892bb44b09be583fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:19.021820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:19.095907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1391/2758] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:19.100472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:19.101851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:19.144328 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:19.173035 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a67a3-b159-4842-b529-6fc653c417d0 service glance] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:19.177517 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:19.178120 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:19.178398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:19.178669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:19.179292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:19.181307 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:19.212251 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1368/2759] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:19 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:19.216799 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:19.228900 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac93e34-4f8b-49b6-a345-346ce53b8830 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:19.288441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:19.288981 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:19.289265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:19.289542 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:19.290832 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:19.292861 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:19.314896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1392/2760] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:19 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:19.319085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:19.330196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fd2fe-8c99-4d02-a115-1ddf1205c695 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:19.334956 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:19.335574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:19.335897 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:19.336174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:19.337577 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:19.339190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:19.350084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1369/2761] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:19.354506 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:19.364820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48eaa3-217a-4df5-973f-5ae133ae71f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:19.369560 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:19.370215 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:19.370499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:19.370776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] PATH_INFO: `/v3/projects/540a39a28c8e413cbe2e2adbaf654cbc/users/26ac57a8a49d44ceb0f2e9439618d98e/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:19.388893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=540a39a28c8e413cbe2e2adbaf654cbc, user_id=26ac57a8a49d44ceb0f2e9439618d98e, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:19.391591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:19.406017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1393/2762] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:19 2022] PUT /identity/v3/projects/540a39a28c8e413cbe2e2adbaf654cbc/users/26ac57a8a49d44ceb0f2e9439618d98e/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:19.409511 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:19.419833 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cef37c-2b4a-4e29-98c4-e5cab3a77422 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:19.514672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:19.515297 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:19.515579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:19.515900 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:19.516550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:19.518139 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:19.530080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1370/2763] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:19.534169 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:19.545857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad391f3-9512-46d3-b421-0c1fd13b3f3b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:19.550557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330cfa9-b3be-4f6f-a4b3-c166420c44bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:19.551245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330cfa9-b3be-4f6f-a4b3-c166420c44bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:19.551527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330cfa9-b3be-4f6f-a4b3-c166420c44bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:19.551843 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330cfa9-b3be-4f6f-a4b3-c166420c44bd admin admin] PATH_INFO: `/v3/projects/540a39a28c8e413cbe2e2adbaf654cbc/users/26ac57a8a49d44ceb0f2e9439618d98e/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:19.553796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330cfa9-b3be-4f6f-a4b3-c166420c44bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=540a39a28c8e413cbe2e2adbaf654cbc, user_id=26ac57a8a49d44ceb0f2e9439618d98e, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:19.557217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330cfa9-b3be-4f6f-a4b3-c166420c44bd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:19.569124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1394/2764] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:19 2022] PUT /identity/v3/projects/540a39a28c8e413cbe2e2adbaf654cbc/users/26ac57a8a49d44ceb0f2e9439618d98e/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:20.585730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:20.587825 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:20.602064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6cc9d-c8b0-488b-b280-d4909e768c68 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:20.675723 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:20.676763 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:20.677250 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:20.677850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] PATH_INFO: `/v3/users/edef859df06b41bfb3bbc85067422f8f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:20.679541 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=edef859df06b41bfb3bbc85067422f8f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:20.682335 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:20.769302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edef859df06b41bfb3bbc85067422f8f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:20.771399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:20.788687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edef859df06b41bfb3bbc85067422f8f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:20.789458 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edef859df06b41bfb3bbc85067422f8f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:20.799401 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edef859df06b41bfb3bbc85067422f8f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:20.822471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edef859df06b41bfb3bbc85067422f8f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:20.831055 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1371/2765] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:20 2022] DELETE /identity/v3/users/edef859df06b41bfb3bbc85067422f8f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:20.836189 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:20.837316 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:20.837428 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:20.867010 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:20.869711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:20.942352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:20.966365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1395/2766] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:20.984508 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:20.987132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:21.086108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994eb5d-0b05-4bc0-b5af-f83682da6c4b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:21.093532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171e877-84b9-4fbd-b6b5-0810cb65bc0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:21.094937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171e877-84b9-4fbd-b6b5-0810cb65bc0d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:21.095231 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171e877-84b9-4fbd-b6b5-0810cb65bc0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:21.095799 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171e877-84b9-4fbd-b6b5-0810cb65bc0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:21.108922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171e877-84b9-4fbd-b6b5-0810cb65bc0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:21.110743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171e877-84b9-4fbd-b6b5-0810cb65bc0d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:21.138085 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1372/2767] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:21.793163 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:21.795308 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:21.834660 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05bdb3-7d20-4e4f-8391-bc6873b04310 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:21.844272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:21.846118 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:21.846624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:21.847235 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] PATH_INFO: `/v3/projects/7b3521a89c164be1be6dd934cfa5d40f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:21.852695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3521a89c164be1be6dd934cfa5d40f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:21.855164 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:21.968358 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3521a89c164be1be6dd934cfa5d40f', 'request_id': 'req-f018e5f3-f1d0-4869-ae01-68552a1922da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:21.980896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3521a89c164be1be6dd934cfa5d40f', 'request_id': 'req-f018e5f3-f1d0-4869-ae01-68552a1922da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:22.010553 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3521a89c164be1be6dd934cfa5d40f', 'request_id': 'req-f018e5f3-f1d0-4869-ae01-68552a1922da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:22.050265 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] The token cache is being invalidate because project 7b3521a89c164be1be6dd934cfa5d40f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:22.050583 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:22.051689 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f018e5f3-f1d0-4869-ae01-68552a1922da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:22.054068 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1396/2768] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:21 2022] DELETE /identity/v3/projects/7b3521a89c164be1be6dd934cfa5d40f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:22.059292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.059949 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.060305 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.086059 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:22.088474 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:22.191573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1373/2769] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:22.200127 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.200619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.201060 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.248736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:22.251007 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:22.268417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1397/2770] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:22.269977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:22.285358 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99885b59-65f9-4d25-81a3-2fc0b3881e6f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:22.294744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:22.295633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.296088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.296550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.297399 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:22.297854 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:22.302520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:22.323190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1374/2771] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:09:22.327278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:22.338752 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5101-e503-49b6-bfad-3f1c1e7f65e0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:22.348306 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:22.348944 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.349228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.349557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.350201 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:22.353062 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:22.394792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1398/2772] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:22 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:22.398653 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:22.420083 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2c8b2-ed33-4daa-9ae4-ad3fd19e74ed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:22.537573 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:22.548979 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:22.549890 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.550376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.551124 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.552319 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:22.554787 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:22.586520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1375/2773] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:22 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:22.590899 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:22.607645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead4c2b-d920-466a-942f-f34098743cbd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:22.613063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:22.613834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.617055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.617055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.617055 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:22.617463 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:22.648884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1399/2774] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:22.653336 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:22.713845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbf236-8d82-40e2-bdcc-a71f6ac74078 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:22.728943 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678d7f6-f620-4c6d-9f4f-fbc57f8657c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:22.729560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678d7f6-f620-4c6d-9f4f-fbc57f8657c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.729839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678d7f6-f620-4c6d-9f4f-fbc57f8657c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.730111 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678d7f6-f620-4c6d-9f4f-fbc57f8657c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.731423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678d7f6-f620-4c6d-9f4f-fbc57f8657c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:22.733080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678d7f6-f620-4c6d-9f4f-fbc57f8657c4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:22.757445 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:22.769378 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1376/2775] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:09:22.777680 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a2fb6-a0e4-4b69-8393-74569f752eb5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:22.786648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:22.788640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.789108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.790253 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] PATH_INFO: `/v3/projects/7a990eeda9fd4830a4b436526804122b/users/69f735eef4a641118c5f8c1a94a90a30/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:22.834703 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a990eeda9fd4830a4b436526804122b, user_id=69f735eef4a641118c5f8c1a94a90a30, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:22.839285 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:22.885492 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1400/2776] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:22 2022] PUT /identity/v3/projects/7a990eeda9fd4830a4b436526804122b/users/69f735eef4a641118c5f8c1a94a90a30/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:22.953175 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f25c8-59ca-4dea-9600-986900547689 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:22.953485 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f25c8-59ca-4dea-9600-986900547689 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:22.953840 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f25c8-59ca-4dea-9600-986900547689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:23.014867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-962f25c8-59ca-4dea-9600-986900547689 None None] MFA Rules not processed for user `69f735eef4a641118c5f8c1a94a90a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:23.016580 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-962f25c8-59ca-4dea-9600-986900547689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:23.191414 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1377/2777] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:23.212841 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:23.215520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:23.357755 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633a9e-bca9-49cb-a65e-377c23c10259 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:23.362905 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:23.363293 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:23.363834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:23.364286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:23.398130 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:23.400390 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:23.421407 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1401/2778] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:25.390390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a005d6-f8e4-4bbd-8a85-fb39c376442b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:25.390843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a005d6-f8e4-4bbd-8a85-fb39c376442b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:25.390941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a005d6-f8e4-4bbd-8a85-fb39c376442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:25.440497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-68a005d6-f8e4-4bbd-8a85-fb39c376442b None None] MFA Rules not processed for user `26ac57a8a49d44ceb0f2e9439618d98e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:25.441733 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-68a005d6-f8e4-4bbd-8a85-fb39c376442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:25.534126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1378/2779] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:28.343171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:28.344634 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:28.356856 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7b4bf-c263-45fb-831c-62dfd882e38f service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:28.360802 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337f5152bd51454cb58d6b2092d5cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:28.361413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:28.361698 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:28.361974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:28.363872 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:28.379571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:28.433273 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:28.434652 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:28.444813 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1402/2780] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:28.596774 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488137d-d056-40e9-8804-cd5a40512497 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:28.599000 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488137d-d056-40e9-8804-cd5a40512497 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:28.599796 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488137d-d056-40e9-8804-cd5a40512497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:28.625663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9488137d-d056-40e9-8804-cd5a40512497 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:28.627534 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9488137d-d056-40e9-8804-cd5a40512497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:28.721411 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:28.730760 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1379/2781] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:09:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:28.781360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:28.782879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:28.860195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ac754-6f10-40c0-a4c9-295af2772729 service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:28.865219 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:28.865824 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:28.866108 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:28.866379 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:28.875181 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:28.877046 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:28.885557 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1403/2782] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:09:37.295189 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915c77-b604-461e-9df9-9feae2685332 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:37.295813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915c77-b604-461e-9df9-9feae2685332 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:37.295813 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915c77-b604-461e-9df9-9feae2685332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:37.314807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-34915c77-b604-461e-9df9-9feae2685332 None None] MFA Rules not processed for user `97ab9c24c5404e1892bb44b09be583fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:37.316038 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-34915c77-b604-461e-9df9-9feae2685332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:37.323435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1380/2783] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:37.581080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:37.582498 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:37.640501 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03913-8416-4aca-be6b-66c362d7808e service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:37.644468 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:37.645212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:37.645531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:37.645884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:37.648150 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:37.657902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:37.659321 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:37.667922 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1404/2784] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:39.506893 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:39.507398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:39.507524 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:39.527521 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:39.528765 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:39.536040 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1381/2785] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 9 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:39.548903 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:39.561764 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03e961-ee6a-4538-94e7-e051fc6744b4 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:39.567309 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1b0dc-0fc4-4a50-bfec-75148135cbf3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:39.567950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b0dc-0fc4-4a50-bfec-75148135cbf3 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:39.568236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b0dc-0fc4-4a50-bfec-75148135cbf3 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:39.568515 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b0dc-0fc4-4a50-bfec-75148135cbf3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:39.577582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1b0dc-0fc4-4a50-bfec-75148135cbf3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:39.579157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1b0dc-0fc4-4a50-bfec-75148135cbf3 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:39.587988 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1405/2786] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:09:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:09:49.521347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=9 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:49.584022 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:57.480330 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:57.481741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:57.546667 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2f6af-e94f-46e2-ad56-23b23665464c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:57.551094 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:57.551746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:57.552024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:57.552295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] PATH_INFO: `/v3/users/8016241368a14156badd1f862f48d5bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:57.553403 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8016241368a14156badd1f862f48d5bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:57.556278 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:57.600922 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8016241368a14156badd1f862f48d5bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:57.608042 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8016241368a14156badd1f862f48d5bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:57.608638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8016241368a14156badd1f862f48d5bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:57.615814 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8016241368a14156badd1f862f48d5bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:57.628932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8016241368a14156badd1f862f48d5bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:57.635420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1382/2787] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:09:57 2022] DELETE /identity/v3/users/8016241368a14156badd1f862f48d5bc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:57.640021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72329d68-80fd-4523-8905-28bb905324dc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:57.640496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72329d68-80fd-4523-8905-28bb905324dc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:57.640596 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72329d68-80fd-4523-8905-28bb905324dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:57.659514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-72329d68-80fd-4523-8905-28bb905324dc None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:57.660746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-72329d68-80fd-4523-8905-28bb905324dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:57.719349 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1406/2788] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:57.729743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:57.731138 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:57.787807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d20db-4d64-4a2d-9f5b-74cf3ed1a12b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:57.792390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f2324-0afb-4d77-b885-42cd3e10d157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:57.792988 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f2324-0afb-4d77-b885-42cd3e10d157 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:57.793262 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f2324-0afb-4d77-b885-42cd3e10d157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:57.793531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f2324-0afb-4d77-b885-42cd3e10d157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:57.802190 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f2324-0afb-4d77-b885-42cd3e10d157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:57.803607 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f2324-0afb-4d77-b885-42cd3e10d157 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:57.813490 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1383/2789] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:09:58.085278 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72329d68-80fd-4523-8905-28bb905324dc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.086815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-72329d68-80fd-4523-8905-28bb905324dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:58.099916 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72329d68-80fd-4523-8905-28bb905324dc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.105482 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.106360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.106817 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.107440 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] PATH_INFO: `/v3/projects/fcd635315d2e40bfb391108a5f4da431` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.109885 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd635315d2e40bfb391108a5f4da431)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.111686 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.164844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd635315d2e40bfb391108a5f4da431', 'request_id': 'req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:58.174032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd635315d2e40bfb391108a5f4da431', 'request_id': 'req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:58.181550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd635315d2e40bfb391108a5f4da431', 'request_id': 'req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:58.193718 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] The token cache is being invalidate because project fcd635315d2e40bfb391108a5f4da431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:09:58.194417 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:58.194669 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06e7f1b4-32d4-47aa-8a95-75daa77b3aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:09:58.197612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1407/2790] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:09:58 2022] DELETE /identity/v3/projects/fcd635315d2e40bfb391108a5f4da431 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:09:58.202141 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.202510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.202857 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.223786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:58.225051 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:58.257589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:58.329327 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1384/2791] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:58.340248 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.340935 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.341388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.368736 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:58.369894 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:58.392403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1408/2792] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:58.396291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.407037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca9fc9-72d2-4726-87b3-ebd9aa50d369 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.411693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.412300 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.412589 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.412862 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.413525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:09:58.413805 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.416234 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.429001 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1385/2793] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:09:58.434106 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.444354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bfb9b-41b8-404f-804c-a88908739b16 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.450850 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.451681 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.452009 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.452317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.453059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.455135 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.489893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1409/2794] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:09:58 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:58.491147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.503797 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887abd-9d67-4bd3-bbb1-691e2791c61f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.570312 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.571248 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.571539 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.571846 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.572465 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.574390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.597581 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1386/2795] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:09:58 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:09:58.601365 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.612994 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df7d93-38a1-46ba-a61a-bcca3106f872 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.617898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.618641 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.619001 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.619220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.620029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.621609 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.632406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1410/2796] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:58.636790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.647200 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464cf46-621d-4367-887c-0151655b5a05 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.652507 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5e5a1-966f-4b77-9e27-0e65d947e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.653108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5e5a1-966f-4b77-9e27-0e65d947e0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.653388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5e5a1-966f-4b77-9e27-0e65d947e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.653660 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5e5a1-966f-4b77-9e27-0e65d947e0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.654311 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5e5a1-966f-4b77-9e27-0e65d947e0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.657812 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5e5a1-966f-4b77-9e27-0e65d947e0aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.676727 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1387/2797] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:09:58.681193 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.691049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32393529-f77f-4eb5-855d-0c091c3233c1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.696194 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.696991 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.697259 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.697525 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] PATH_INFO: `/v3/projects/cf6e58abe0054b2b84b1ef26b142cb01/users/ee0115f80e034d79b42123e846fe70b6/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.712452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6e58abe0054b2b84b1ef26b142cb01, user_id=ee0115f80e034d79b42123e846fe70b6, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.715103 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.727765 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1411/2798] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:09:58 2022] PUT /identity/v3/projects/cf6e58abe0054b2b84b1ef26b142cb01/users/ee0115f80e034d79b42123e846fe70b6/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:09:58.737884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.738169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.738440 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.766605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] MFA Rules not processed for user `ee0115f80e034d79b42123e846fe70b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:09:58.767843 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:58.831196 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1388/2799] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:09:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:09:58.843402 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:09:58.845144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:09:58.867227 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:09:58.905213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfd259-10b2-4276-8868-efadfde62864 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:09:58.908777 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1174ac-e66f-4e28-bc13-8195d84cc401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:09:58.909374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1174ac-e66f-4e28-bc13-8195d84cc401 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:09:58.909648 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1174ac-e66f-4e28-bc13-8195d84cc401 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:09:58.909917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1174ac-e66f-4e28-bc13-8195d84cc401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:09:58.918779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1174ac-e66f-4e28-bc13-8195d84cc401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:09:58.920249 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1174ac-e66f-4e28-bc13-8195d84cc401 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:09:58.928738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1412/2800] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:09:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:01.706870 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:01.708302 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:01.728750 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:01.776109 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ceb72-229e-4eb5-b08a-7deae9a85bb3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:01.780575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:01.781204 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:01.781481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:01.781756 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] PATH_INFO: `/v3/users/69f735eef4a641118c5f8c1a94a90a30` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:01.782881 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f735eef4a641118c5f8c1a94a90a30)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:01.784816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:01.833904 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f735eef4a641118c5f8c1a94a90a30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:01.846794 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f735eef4a641118c5f8c1a94a90a30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:01.847469 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f735eef4a641118c5f8c1a94a90a30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:01.855992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f735eef4a641118c5f8c1a94a90a30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:01.870310 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f735eef4a641118c5f8c1a94a90a30'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:01.876913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1389/2801] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:10:01 2022] DELETE /identity/v3/users/69f735eef4a641118c5f8c1a94a90a30 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:10:01.881779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:01.882361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:01.882565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:01.901396 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:01.902600 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:01.965395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1413/2802] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:10:01.975821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:01.977795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:02.040996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d20bd-26e3-4e7a-88fb-7fce1e48122f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:02.044925 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:02.045619 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:02.045906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:02.046180 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:02.055341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:02.056850 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:02.066663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1390/2803] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:02.355697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:02.357441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:02.371048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfef619-1e2b-49fd-88e3-dd9be2c15d8f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:02.376204 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:02.377214 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:02.377599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:02.378011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] PATH_INFO: `/v3/projects/7a990eeda9fd4830a4b436526804122b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:02.379236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a990eeda9fd4830a4b436526804122b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:02.381294 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:02.429787 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a990eeda9fd4830a4b436526804122b', 'request_id': 'req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:02.436919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a990eeda9fd4830a4b436526804122b', 'request_id': 'req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:02.443031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a990eeda9fd4830a4b436526804122b', 'request_id': 'req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:02.454395 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] The token cache is being invalidate because project 7a990eeda9fd4830a4b436526804122b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:10:02.454682 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:02.455317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:02.456792 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1414/2804] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:10:02 2022] DELETE /identity/v3/projects/7a990eeda9fd4830a4b436526804122b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:10:09.692347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:09.701066 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00012c9-e7c5-4969-8157-a3878d15790b service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:09.738296 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:09.750584 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:09.751179 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:09.751758 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:09.752027 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:09.753741 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:09.813672 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:09.815075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:09.824280 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1391/2805] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:12.451373 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:13.013045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:13.014603 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:13.118510 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5f8ed-31b4-4e9a-97bc-24b27ff70198 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:13.124274 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:13.124898 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:13.125183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:13.125455 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:13.127626 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:13.143372 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:13.145049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f510da-558a-40d7-9023-8ce08a6a4cb9 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:13.158334 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1415/2806] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:19.820301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=32 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:23.152711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=34 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:32.958516 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:32.959978 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:33.085815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d52bc-3585-4fbe-b59c-34686028b661 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:33.090740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:33.091373 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.091740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.092013 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] PATH_INFO: `/v3/users/ee0115f80e034d79b42123e846fe70b6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.093182 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0115f80e034d79b42123e846fe70b6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:33.095132 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:33.141412 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0115f80e034d79b42123e846fe70b6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.148191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0115f80e034d79b42123e846fe70b6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.148839 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0115f80e034d79b42123e846fe70b6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.156052 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0115f80e034d79b42123e846fe70b6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.168054 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0115f80e034d79b42123e846fe70b6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.175590 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1392/2807] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:10:32 2022] DELETE /identity/v3/users/ee0115f80e034d79b42123e846fe70b6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:10:33.180821 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.181336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.181447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.209019 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:33.210228 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:33.267902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1416/2808] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:10:33.277982 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:33.279390 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:33.300214 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:33.354597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e07644-94c3-4e14-98fe-2ceae17007bc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:33.358917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d586be-ec30-411b-b3c5-d5b300eb8a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:33.359515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d586be-ec30-411b-b3c5-d5b300eb8a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.359822 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d586be-ec30-411b-b3c5-d5b300eb8a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.360092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d586be-ec30-411b-b3c5-d5b300eb8a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.368831 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d586be-ec30-411b-b3c5-d5b300eb8a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:33.370238 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d586be-ec30-411b-b3c5-d5b300eb8a3f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:33.380707 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1393/2809] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:33.646591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:33.648198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:33.663017 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bf1cb-9a13-427f-8b77-17e492faabed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:33.668433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:33.669146 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.669424 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.669693 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] PATH_INFO: `/v3/projects/cf6e58abe0054b2b84b1ef26b142cb01` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.670815 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6e58abe0054b2b84b1ef26b142cb01)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:33.673761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:33.721437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6e58abe0054b2b84b1ef26b142cb01', 'request_id': 'req-bcf4054a-9b46-479a-8653-4db3fdb4e505'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.728711 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6e58abe0054b2b84b1ef26b142cb01', 'request_id': 'req-bcf4054a-9b46-479a-8653-4db3fdb4e505'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.735854 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6e58abe0054b2b84b1ef26b142cb01', 'request_id': 'req-bcf4054a-9b46-479a-8653-4db3fdb4e505'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.745954 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] The token cache is being invalidate because project cf6e58abe0054b2b84b1ef26b142cb01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:10:33.746243 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.746879 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bcf4054a-9b46-479a-8653-4db3fdb4e505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:10:33.748388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1417/2810] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:10:33 2022] DELETE /identity/v3/projects/cf6e58abe0054b2b84b1ef26b142cb01 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:10:33.753618 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.753913 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.754191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.803069 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:33.804292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:33.864603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1394/2811] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:10:33.873556 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06887761-add0-439e-af73-3f1e85109432 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.873842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06887761-add0-439e-af73-3f1e85109432 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.874114 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06887761-add0-439e-af73-3f1e85109432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.894091 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-06887761-add0-439e-af73-3f1e85109432 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:33.895280 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-06887761-add0-439e-af73-3f1e85109432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:33.903659 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1418/2812] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:10:33.907992 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:33.917636 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efa01d-4831-4377-90c8-241c4f1bc8d7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:33.922151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:33.922743 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.923022 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.923294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.923973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:10:33.924277 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:33.925860 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:33.937288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1395/2813] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:10:33.941499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06887761-add0-439e-af73-3f1e85109432 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:33.951420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06887761-add0-439e-af73-3f1e85109432 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:33.956133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:33.956731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:33.957013 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:33.957288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:33.957919 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:33.959833 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:33.988355 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1419/2814] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:10:33 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:10:33.992510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:34.002697 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d07ca-f943-44b2-8a6a-d0ad55f86cf3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:34.057603 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:34.058260 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:34.058631 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:34.058911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:34.059515 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:34.061445 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:34.078400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:34.086211 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1396/2815] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:10:33 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:10:34.090716 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:34.100946 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6ae2-3eda-4e11-8e77-bfd24c3ce985 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:34.105485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:34.106073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:34.106348 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:34.106615 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:34.107246 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:34.108855 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:34.118434 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1420/2816] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:10:34.122452 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:34.132418 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369e30-9414-457c-9743-b32c01a7ef48 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:34.137766 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca639f-3948-494a-8856-54594d139fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:34.138357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca639f-3948-494a-8856-54594d139fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:34.138633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca639f-3948-494a-8856-54594d139fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:34.138905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca639f-3948-494a-8856-54594d139fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:34.139538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca639f-3948-494a-8856-54594d139fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:34.141149 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca639f-3948-494a-8856-54594d139fe6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:34.153442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1397/2817] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:10:34.157435 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:34.167122 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9437d-1bdc-4067-8f8c-4a11edfd83ef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:34.171913 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:34.172550 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:34.172825 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:34.173093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] PATH_INFO: `/v3/projects/bd477a3a558642539d0bce76c8d505f3/users/b608ef770dec44b489230d044bf5892f/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:34.187958 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd477a3a558642539d0bce76c8d505f3, user_id=b608ef770dec44b489230d044bf5892f, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:34.190564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:34.204148 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1421/2818] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:10:34 2022] PUT /identity/v3/projects/bd477a3a558642539d0bce76c8d505f3/users/b608ef770dec44b489230d044bf5892f/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:10:34.209108 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d43306-c0eb-446e-867f-1a7348693c2b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:34.209394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d43306-c0eb-446e-867f-1a7348693c2b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:34.209695 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d43306-c0eb-446e-867f-1a7348693c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:34.228333 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-67d43306-c0eb-446e-867f-1a7348693c2b None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:34.229532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-67d43306-c0eb-446e-867f-1a7348693c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:34.293357 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1398/2819] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:10:34.304015 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:34.305734 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:34.320389 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:34.363993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f03786-36b1-416a-96e1-1df395e73689 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:34.367617 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:34.368229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:34.368509 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:34.368780 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:34.378507 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:34.379987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:34.390392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1422/2820] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:35.005435 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff57b-f5f3-46b5-a58f-422560f9009b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:35.005946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff57b-f5f3-46b5-a58f-422560f9009b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:35.006084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff57b-f5f3-46b5-a58f-422560f9009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:35.034513 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4f9ff57b-f5f3-46b5-a58f-422560f9009b None None] MFA Rules not processed for user `b608ef770dec44b489230d044bf5892f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:35.035798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4f9ff57b-f5f3-46b5-a58f-422560f9009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:35.102749 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1399/2821] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:10:35.114785 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:35.116250 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:35.176828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712bc30-ced4-48ec-968f-99624099c631 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:35.181644 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:35.182288 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:35.182571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:35.182846 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:35.191691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:35.193138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:35.202190 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1423/2822] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:36.557538 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:36.559332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:36.559332 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:36.579855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] MFA Rules not processed for user `b608ef770dec44b489230d044bf5892f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:10:36.581072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:36.588226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1400/2823] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:10:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:10:36.833744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:10:36.835156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:36.848433 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903947ad-eb12-4644-8f76-996dc94ec2f1 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:10:36.851975 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:10:36.852705 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:10:36.853109 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:10:36.853386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:10:36.855153 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:10:36.865192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:10:36.866602 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3c136-cc28-4edb-b244-3131074a50a9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:10:36.874747 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1424/2824] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:10:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:10:46.573740 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=68 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:10:46.870852 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=52 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:11:27.636076 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:27.637501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:27.728482 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276b40c-45c0-402b-a8f1-ab92dd9c8a8a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:27.733472 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:27.734075 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:27.734359 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:27.734634 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:27.736398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:27.748176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:27.749748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fe871-b76f-4529-896a-9a7f6c1bc36d service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:27.759828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1401/2825] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:11:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:11:36.439242 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:36.439809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:36.439809 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:36.459437 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] MFA Rules not processed for user `b608ef770dec44b489230d044bf5892f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:11:36.461352 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:36.468639 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1425/2826] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:11:37.755700 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=41 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:11:42.984331 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:42.984883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:42.984883 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:43.016329 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] MFA Rules not processed for user `4dd6c53afc054e2384dc0d247a9b935c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:11:43.017625 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:43.080172 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1402/2827] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 9 07:11:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:11:43.123613 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:43.125067 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:43.202100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aaf30-6bf6-4f2c-806c-93bbff9062cc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:43.208162 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f3a4e99f3044d35aa2e310478f3653b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:43.208776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:43.209056 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:43.209330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:43.217943 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:43.219511 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:43.228073 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1426/2828] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 9 07:11:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 07:11:47.001512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:47.003126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:47.018781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d3a8e-556d-47a7-a7a8-7aeafe50e553 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:47.028821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:47.029522 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:47.029793 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:47.030169 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:47.032346 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:47.048147 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:47.049848 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:47.077873 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1403/2829] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:11:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:11:50.305173 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:50.306605 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:50.321820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c5d1-cd40-4828-9b0a-dab668d5f27c service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:50.326439 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:50.327063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:50.327344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:50.327640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] PATH_INFO: `/v3/users/97ab9c24c5404e1892bb44b09be583fe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:50.328760 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ab9c24c5404e1892bb44b09be583fe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:50.330656 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:50.379831 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ab9c24c5404e1892bb44b09be583fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.388632 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ab9c24c5404e1892bb44b09be583fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.397541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ab9c24c5404e1892bb44b09be583fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.398247 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ab9c24c5404e1892bb44b09be583fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.410612 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ab9c24c5404e1892bb44b09be583fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.420221 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1427/2830] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:11:50 2022] DELETE /identity/v3/users/97ab9c24c5404e1892bb44b09be583fe => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:11:50.592945 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:50.605616 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952146a-1783-43c4-9834-1de6f27256d5 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:50.665301 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:50.665932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:50.666216 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:50.666493 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] PATH_INFO: `/v3/projects/272a737decfd4dba9538dffbc7c1db5b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:50.667680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] RBAC: Authorizing `identity:delete_project(project_id=272a737decfd4dba9538dffbc7c1db5b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:50.669790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:50.715903 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272a737decfd4dba9538dffbc7c1db5b', 'request_id': 'req-41e1ffd3-e721-424e-8a72-d595871d187b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.721268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272a737decfd4dba9538dffbc7c1db5b', 'request_id': 'req-41e1ffd3-e721-424e-8a72-d595871d187b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.727527 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272a737decfd4dba9538dffbc7c1db5b', 'request_id': 'req-41e1ffd3-e721-424e-8a72-d595871d187b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.739415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] The token cache is being invalidate because project 272a737decfd4dba9538dffbc7c1db5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:11:50.739739 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.740379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.741977 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1404/2831] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:11:50 2022] DELETE /identity/v3/projects/272a737decfd4dba9538dffbc7c1db5b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:11:50.745968 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:50.747392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:50.761491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa67c-5f97-45d0-a37f-7f7c3a1a52e0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:50.810624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:11:50.819289 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:50.820014 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:50.820546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:50.821033 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] PATH_INFO: `/v3/users/26ac57a8a49d44ceb0f2e9439618d98e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:50.822363 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ac57a8a49d44ceb0f2e9439618d98e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:50.824374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:50.871873 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ac57a8a49d44ceb0f2e9439618d98e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.879400 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ac57a8a49d44ceb0f2e9439618d98e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.887070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ac57a8a49d44ceb0f2e9439618d98e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.887874 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ac57a8a49d44ceb0f2e9439618d98e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.899945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ac57a8a49d44ceb0f2e9439618d98e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:50.905692 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1428/2832] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:11:50 2022] DELETE /identity/v3/users/26ac57a8a49d44ceb0f2e9439618d98e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:11:51.079935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:11:51.081395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:11:51.098768 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1ffd3-e721-424e-8a72-d595871d187b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:11:51.105698 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:11:51.166079 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:11:51.166931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:11:51.167369 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:11:51.167684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] PATH_INFO: `/v3/projects/540a39a28c8e413cbe2e2adbaf654cbc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:11:51.168817 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=540a39a28c8e413cbe2e2adbaf654cbc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:11:51.170705 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:11:51.218687 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '540a39a28c8e413cbe2e2adbaf654cbc', 'request_id': 'req-6f3f5353-7a0f-4d21-9535-af2a326a8803'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:51.224006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '540a39a28c8e413cbe2e2adbaf654cbc', 'request_id': 'req-6f3f5353-7a0f-4d21-9535-af2a326a8803'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:51.230905 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '540a39a28c8e413cbe2e2adbaf654cbc', 'request_id': 'req-6f3f5353-7a0f-4d21-9535-af2a326a8803'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:51.243495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] The token cache is being invalidate because project 540a39a28c8e413cbe2e2adbaf654cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:11:51.243809 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:51.244425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:11:51.245869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1405/2833] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:11:51 2022] DELETE /identity/v3/projects/540a39a28c8e413cbe2e2adbaf654cbc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:12:00.902284 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:01.239941 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:30.751272 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:30.753591 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:30.802720 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:30.804255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:30.833082 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228889e-7556-4297-abdc-1d922aa53e4a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:30.874916 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f5353-7a0f-4d21-9535-af2a326a8803 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:30.879917 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:30.880553 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:30.880837 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:30.881114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] PATH_INFO: `/v3/users/b608ef770dec44b489230d044bf5892f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:30.882795 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b608ef770dec44b489230d044bf5892f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:30.884759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:30.898541 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:30.899184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:30.899461 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:30.899763 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:30.901726 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:30.914102 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:30.915767 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:30.926100 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1429/2834] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:12:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:12:30.931894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b608ef770dec44b489230d044bf5892f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:30.938926 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b608ef770dec44b489230d044bf5892f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:30.939715 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b608ef770dec44b489230d044bf5892f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:30.948114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b608ef770dec44b489230d044bf5892f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:30.962790 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bb8e1ad3-f9a2-4a33-bd69-f96c583222b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b608ef770dec44b489230d044bf5892f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:30.968702 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1406/2835] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:12:30 2022] DELETE /identity/v3/users/b608ef770dec44b489230d044bf5892f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:12:31.144412 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.165072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29120e-a2a7-4803-8b54-69e75db4ac49 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.222840 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.223480 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.223804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.224087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] PATH_INFO: `/v3/projects/bd477a3a558642539d0bce76c8d505f3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.225209 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd477a3a558642539d0bce76c8d505f3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.227893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.266529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:31.277812 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd477a3a558642539d0bce76c8d505f3', 'request_id': 'req-0bd7d3d2-3781-4e64-a97d-4004acb908a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:31.284362 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd477a3a558642539d0bce76c8d505f3', 'request_id': 'req-0bd7d3d2-3781-4e64-a97d-4004acb908a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:31.291251 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd477a3a558642539d0bce76c8d505f3', 'request_id': 'req-0bd7d3d2-3781-4e64-a97d-4004acb908a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:31.301607 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] The token cache is being invalidate because project bd477a3a558642539d0bce76c8d505f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:12:31.301896 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:31.302529 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd7d3d2-3781-4e64-a97d-4004acb908a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:12:31.304045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1430/2836] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:12:31 2022] DELETE /identity/v3/projects/bd477a3a558642539d0bce76c8d505f3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:12:31.309086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.309380 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.309662 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.328884 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:12:31.330086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:31.388779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1407/2837] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:12:31.396393 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.396677 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.396977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.417624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:12:31.418819 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:31.426590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1431/2838] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:12:31.430728 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.440416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5261b-ab75-491e-b489-18810b259552 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.445126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.445724 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.446007 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.446288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.446936 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:12:31.447242 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.448886 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.460295 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1408/2839] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 07:12:31.465002 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.474131 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34833e2a-3442-4839-8b50-305c5d3878e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.479426 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.480061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.480344 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.480618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.481236 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.483151 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.518462 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1432/2840] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:12:31 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:12:31.522608 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.533012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479fb25-73d2-40fe-b91b-4ffe32bb83d1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.536579 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:31.591612 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.592222 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.592501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.592777 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.593398 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.595309 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.615281 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1409/2841] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:12:31 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:12:31.619545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.630089 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39f773-03ed-4441-abbd-0c3b33c4af67 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.635152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.635788 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.636069 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.636341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.636979 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.638551 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.649562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1433/2842] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:12:31.652691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.663288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28357c-69db-4bfc-b04f-25fa5c45dc66 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.668031 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df547d2-10f1-4811-93cd-aa657a0c0e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.668627 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df547d2-10f1-4811-93cd-aa657a0c0e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.668912 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df547d2-10f1-4811-93cd-aa657a0c0e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.669187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df547d2-10f1-4811-93cd-aa657a0c0e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.669834 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df547d2-10f1-4811-93cd-aa657a0c0e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.671423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df547d2-10f1-4811-93cd-aa657a0c0e12 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.682058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1410/2843] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:12:31.686188 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.696584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b2e08-7dec-4a65-bb20-9fd733e7ef72 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.701156 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.701902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.702072 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.702346 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] PATH_INFO: `/v3/projects/5577bdb690df49f899cfa0f47ad5ba5c/users/062a94c1e6be4695b9758287aa82ab41/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.717808 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5577bdb690df49f899cfa0f47ad5ba5c, user_id=062a94c1e6be4695b9758287aa82ab41, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.720457 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.733133 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1434/2844] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:12:31 2022] PUT /identity/v3/projects/5577bdb690df49f899cfa0f47ad5ba5c/users/062a94c1e6be4695b9758287aa82ab41/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:12:31.738082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cff12-a9db-4351-9957-0c418470e8f7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.738372 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cff12-a9db-4351-9957-0c418470e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.738649 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cff12-a9db-4351-9957-0c418470e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.757895 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-731cff12-a9db-4351-9957-0c418470e8f7 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:12:31.759084 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-731cff12-a9db-4351-9957-0c418470e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:31.815582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1411/2845] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:12:31.825370 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:31.827134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:31.904420 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9c89b-ea60-4486-b4da-1a918654886c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:31.908000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:31.908593 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:31.908871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:31.909139 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:31.918499 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:31.919936 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:31.929198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1435/2846] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:12:35.876821 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:35.877324 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:35.877423 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:35.907721 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] MFA Rules not processed for user `062a94c1e6be4695b9758287aa82ab41`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:12:35.908954 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:35.977058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1412/2847] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:12:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 07:12:35.981658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:35.992041 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77d01-5bce-45c4-b72f-ad0edff8b108 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:35.996775 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:35.997388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:35.997673 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:35.997950 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:35.998579 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:36.000797 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:36.031695 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1436/2848] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:12:35 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:12:36.035623 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:36.047101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6d2ae-878b-4cec-bbe7-2d544eb456a6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:36.059633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:36.107323 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:36.107951 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:36.108230 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:36.108501 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:36.109115 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:36.111006 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:36.131872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1413/2849] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:12:36 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:12:36.136645 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:36.146997 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9135d-df81-402f-88f8-8078c1f6e678 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:36.151691 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:36.152300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:36.152578 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:36.152853 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:36.153500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:36.155086 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:36.160397 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:36.165791 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1437/2850] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:12:36.169946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:36.180865 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8865e6-3b44-418d-9c81-01229d905ac0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:36.185497 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:36.186151 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:36.186431 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:36.186706 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] PATH_INFO: `/v3/projects/c49430ad7afc451aa61ee26a8d499418/users/e1739d696a6b4e49b8627d2e84c24360/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:36.201976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49430ad7afc451aa61ee26a8d499418, user_id=e1739d696a6b4e49b8627d2e84c24360, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:36.204691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:36.217078 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1414/2851] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:12:36 2022] PUT /identity/v3/projects/c49430ad7afc451aa61ee26a8d499418/users/e1739d696a6b4e49b8627d2e84c24360/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:12:36.221080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:36.231070 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3b632-c1eb-4540-b738-0fb523f22781 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:36.288040 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ac278-4c76-403a-beb0-fc2b0ad39fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:36.288781 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ac278-4c76-403a-beb0-fc2b0ad39fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:36.289063 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ac278-4c76-403a-beb0-fc2b0ad39fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:36.289338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ac278-4c76-403a-beb0-fc2b0ad39fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:36.289986 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ac278-4c76-403a-beb0-fc2b0ad39fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:36.291585 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ac278-4c76-403a-beb0-fc2b0ad39fd9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:36.301430 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1438/2852] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:12:36.306450 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:36.316575 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27cc4f-2fa2-461f-aaef-41b2c6e30cf8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:36.321908 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:36.322565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:36.322845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:36.323146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] PATH_INFO: `/v3/projects/c49430ad7afc451aa61ee26a8d499418/users/e1739d696a6b4e49b8627d2e84c24360/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:36.325088 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49430ad7afc451aa61ee26a8d499418, user_id=e1739d696a6b4e49b8627d2e84c24360, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:36.327747 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:36.337755 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1415/2853] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:12:36 2022] PUT /identity/v3/projects/c49430ad7afc451aa61ee26a8d499418/users/e1739d696a6b4e49b8627d2e84c24360/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:12:40.606124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a3c03-78e1-4c38-afa8-3f21f5900f7d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:40.606610 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a3c03-78e1-4c38-afa8-3f21f5900f7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:40.606697 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a3c03-78e1-4c38-afa8-3f21f5900f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:40.641197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e84a3c03-78e1-4c38-afa8-3f21f5900f7d None None] MFA Rules not processed for user `e1739d696a6b4e49b8627d2e84c24360`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:12:40.642562 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e84a3c03-78e1-4c38-afa8-3f21f5900f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:40.731183 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1439/2854] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:12:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:12:40.747103 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:40.749468 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:40.879911 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca1af3-445d-4476-bc28-f57a794ab4db admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:40.888442 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70368822-5e07-4090-8eda-227481d0645b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:40.889080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70368822-5e07-4090-8eda-227481d0645b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:40.889364 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70368822-5e07-4090-8eda-227481d0645b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:40.889640 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70368822-5e07-4090-8eda-227481d0645b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:40.942780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:40.961237 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70368822-5e07-4090-8eda-227481d0645b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:40.962876 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70368822-5e07-4090-8eda-227481d0645b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:40.972420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1416/2855] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:12:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:12:41.964907 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:41.965628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:41.965628 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:41.985563 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] MFA Rules not processed for user `062a94c1e6be4695b9758287aa82ab41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:12:41.987618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:41.994724 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1440/2856] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:12:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:12:50.968980 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:51.979519 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=66 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:12:56.217879 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70368822-5e07-4090-8eda-227481d0645b service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:56.219293 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-70368822-5e07-4090-8eda-227481d0645b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:56.295395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70368822-5e07-4090-8eda-227481d0645b service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:56.298997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d784c96b044ef8b383767e8c1cb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:56.299605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:56.299902 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:56.300176 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:56.301867 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:56.315505 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:56.316934 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:56.328198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1417/2857] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:12:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:12:57.060300 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:12:57.116351 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdff4-5756-4b8c-a3fd-c2517efef269 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:12:57.120144 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:12:57.120758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:12:57.121038 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:12:57.121338 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:12:57.123750 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:12:57.138030 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:12:57.139473 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:12:57.150592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1441/2858] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:13:00.456509 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:13:00.458049 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:13:00.470229 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190744-87e3-490b-8855-4ffb40fae8ec service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:13:00.473803 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:13:00.474404 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:13:00.474683 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:13:00.474950 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:13:00.476614 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:13:00.485933 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:13:00.487347 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:13:00.495463 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1418/2859] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:13:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:13:07.145546 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=26 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:13:10.491824 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=41 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:13:11.369810 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:13:11.379004 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f0d55-4df3-41a7-9845-4162baec6967 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:13:11.382564 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:13:11.383157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:13:11.383443 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:13:11.383758 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:13:11.392768 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:13:11.394196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:13:11.404087 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1442/2860] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:13:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:13:21.399382 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:15:21.621425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:15:21.630597 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4e30e-57aa-48cd-973c-a41350d6bce4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:15:21.634459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:15:21.635128 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:15:21.635528 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:15:21.635859 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:15:21.637638 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:15:21.647932 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:15:21.649437 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:15:21.659565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1419/2861] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:15:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:15:21.775945 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:15:21.784762 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff9cc9-9291-4293-aa0c-bb0619210b88 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:15:21.788629 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:15:21.789325 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:15:21.789675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:15:21.790029 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:15:21.791917 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:15:21.806025 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:15:21.807540 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:15:21.819893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1443/2862] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:15:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:15:21.880691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:15:21.882099 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:15:21.949455 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612dc1-3ed3-4c9f-9b8c-d370963358ee service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:15:21.954351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:15:21.954970 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:15:21.955279 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:15:21.955550 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:15:21.957285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:15:21.968592 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:15:21.970146 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:15:21.980210 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1420/2863] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:15:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:15:23.843902 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:15:23.845530 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:15:23.860330 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6c98e-ba3d-4668-9395-42ba582d8008 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:15:23.866276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:15:23.866904 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:15:23.867210 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:15:23.867456 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:15:23.869207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:15:23.882421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:15:23.884021 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:15:23.894531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1444/2864] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:15:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:15:31.976037 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=38 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:15:33.890049 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=12 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:16:47.472658 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:47.488397 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103382ba-70da-4cef-b73b-f864b54e5463 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:47.495064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:47.495694 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:47.495976 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:47.496247 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:47.497997 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:47.512562 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:47.514164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:47.527218 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1421/2865] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:16:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:16:49.452413 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:49.453925 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:49.468834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca76d-18f1-43ea-aafa-867da436f901 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:49.474500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:49.474500 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:49.475317 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:49.475571 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:49.477441 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:49.489256 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:49.491744 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:49.502356 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1445/2866] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:16:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:16:49.877354 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:49.889800 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f730d57-ac0d-495f-ba48-c8b21489cc7b service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:49.895560 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:49.896198 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:49.896477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:49.897400 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:49.899291 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:49.912091 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:49.913816 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:49.924713 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1422/2867] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:16:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:16:54.301938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:54.313568 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db31ef-1e7f-4701-bb39-2aa5ff4bb845 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:54.319134 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:54.319779 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:54.320061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:54.320336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:54.322093 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:54.334061 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:54.335675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:54.347335 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1446/2868] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:16:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:16:57.477780 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:57.541990 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd19ea0-a9fa-4d59-8ee1-fc7c203aa0d2 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:57.546716 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:57.547349 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:57.547675 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:57.547955 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] PATH_INFO: `/v3/users/062a94c1e6be4695b9758287aa82ab41` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:57.549086 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=062a94c1e6be4695b9758287aa82ab41)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:57.550996 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:57.595937 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062a94c1e6be4695b9758287aa82ab41'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.602746 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062a94c1e6be4695b9758287aa82ab41'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.603408 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062a94c1e6be4695b9758287aa82ab41'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.610679 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062a94c1e6be4695b9758287aa82ab41'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.622556 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062a94c1e6be4695b9758287aa82ab41'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.628537 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1423/2869] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:16:57 2022] DELETE /identity/v3/users/062a94c1e6be4695b9758287aa82ab41 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:16:57.805307 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:57.819006 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dd1e4-d9a5-43e9-9b4d-e00ed03deddc service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:57.881746 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:57.882384 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:57.882665 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:57.882940 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] PATH_INFO: `/v3/projects/5577bdb690df49f899cfa0f47ad5ba5c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:57.884080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5577bdb690df49f899cfa0f47ad5ba5c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:57.886000 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:57.932178 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5577bdb690df49f899cfa0f47ad5ba5c', 'request_id': 'req-a5c290b8-536f-4582-9b2e-baa30b81a337'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.939185 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5577bdb690df49f899cfa0f47ad5ba5c', 'request_id': 'req-a5c290b8-536f-4582-9b2e-baa30b81a337'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.945837 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5577bdb690df49f899cfa0f47ad5ba5c', 'request_id': 'req-a5c290b8-536f-4582-9b2e-baa30b81a337'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.955966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] The token cache is being invalidate because project 5577bdb690df49f899cfa0f47ad5ba5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:16:57.956254 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.956887 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:57.958360 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1447/2870] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:16:57 2022] DELETE /identity/v3/projects/5577bdb690df49f899cfa0f47ad5ba5c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:16:57.962915 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:57.964425 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:57.978002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324b48-1cb1-4720-94ab-11c093548d00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.026191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:16:58.037080 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.038294 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.038574 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.038847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] PATH_INFO: `/v3/users/e1739d696a6b4e49b8627d2e84c24360` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.040028 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1739d696a6b4e49b8627d2e84c24360)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.041973 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.086026 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1739d696a6b4e49b8627d2e84c24360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.092919 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1739d696a6b4e49b8627d2e84c24360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.093596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1739d696a6b4e49b8627d2e84c24360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.101208 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1739d696a6b4e49b8627d2e84c24360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.113101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-af898803-ee9d-47d3-a494-4094ffa5a271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1739d696a6b4e49b8627d2e84c24360'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.118960 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1424/2871] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:16:57 2022] DELETE /identity/v3/users/e1739d696a6b4e49b8627d2e84c24360 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:16:58.291969 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.293361 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:58.306517 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c290b8-536f-4582-9b2e-baa30b81a337 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.363759 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.364392 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.364674 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.364948 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] PATH_INFO: `/v3/projects/c49430ad7afc451aa61ee26a8d499418` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.366738 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49430ad7afc451aa61ee26a8d499418)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.368676 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.374776 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:16:58.416094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49430ad7afc451aa61ee26a8d499418', 'request_id': 'req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.423160 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49430ad7afc451aa61ee26a8d499418', 'request_id': 'req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.429756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49430ad7afc451aa61ee26a8d499418', 'request_id': 'req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.440385 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] The token cache is being invalidate because project c49430ad7afc451aa61ee26a8d499418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:16:58.440672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.441333 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8c05ecc9-b2c0-43fe-8f3b-054a80d8317d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:16:58.442909 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1448/2872] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:16:58 2022] DELETE /identity/v3/projects/c49430ad7afc451aa61ee26a8d499418 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:16:58.448233 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.448525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.448806 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.467989 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:16:58.469195 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:58.531263 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1425/2873] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:16:58.539587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.539906 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.540184 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.561374 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:16:58.562592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:58.570937 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1449/2874] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:16:58.575351 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.586039 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8811fc9-c8de-4611-811a-2053e794da8d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.590656 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.591258 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.591565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.591868 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.592510 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 07:16:58.592810 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.594432 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.607385 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1426/2875] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 9 07:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 07:16:58.611491 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.620980 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46a5-20d7-4c74-8b0c-05e245053909 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.625531 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.626132 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.626406 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.626670 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.627286 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.629212 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.657858 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1450/2876] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:16:58 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:16:58.662783 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.672935 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb140de3-d8ad-48f3-9d17-1e5fa81c3838 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.729255 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.729855 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.730136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.730410 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.731033 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.732974 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.740828 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:16:58.756002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1427/2877] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:16:58 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:16:58.760045 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.770283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37881f-48c1-4a14-a073-3d5fabadcba4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.775446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.776085 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.776366 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.776640 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.777282 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.778880 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.789453 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1451/2878] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:16:58.793226 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.803002 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023bb2a-9a09-470d-bb7f-d07d8f73465c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.807578 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03ba5c-92e4-468c-bf47-d01be0661ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.808191 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ba5c-92e4-468c-bf47-d01be0661ced admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.808471 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ba5c-92e4-468c-bf47-d01be0661ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.808742 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ba5c-92e4-468c-bf47-d01be0661ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.809376 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03ba5c-92e4-468c-bf47-d01be0661ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.810946 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03ba5c-92e4-468c-bf47-d01be0661ced admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.821253 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1428/2879] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:16:58.825138 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.835549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c381b0-3413-4647-b728-183b7b759822 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:58.840260 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:58.840920 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.841206 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.841485 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] PATH_INFO: `/v3/projects/ac9f07b87ad643b58d21be9042b64c87/users/cc228ac26c454318b15e1a2f198ebdf4/roles/036ddec2546a4fe7bd3b8eca42111ba0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.857675 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f07b87ad643b58d21be9042b64c87, user_id=cc228ac26c454318b15e1a2f198ebdf4, role_id=036ddec2546a4fe7bd3b8eca42111ba0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:58.860829 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:58.873359 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1452/2880] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:16:58 2022] PUT /identity/v3/projects/ac9f07b87ad643b58d21be9042b64c87/users/cc228ac26c454318b15e1a2f198ebdf4/roles/036ddec2546a4fe7bd3b8eca42111ba0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:16:58.878520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7132-680e-4d78-a2ea-e12ff162efdd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:58.878811 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7132-680e-4d78-a2ea-e12ff162efdd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:58.879089 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7132-680e-4d78-a2ea-e12ff162efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:58.898807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2beb7132-680e-4d78-a2ea-e12ff162efdd None None] MFA Rules not processed for user `b26927358e664fe3981d02119bbe804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:16:58.900050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2beb7132-680e-4d78-a2ea-e12ff162efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:58.956898 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1429/2881] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:16:58.967388 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:16:58.969195 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:16:59.026090 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c8a0-0f05-4191-bc82-62b98a709fed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:16:59.029672 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:16:59.030276 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:16:59.030559 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:16:59.030834 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:16:59.039976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:16:59.041401 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:16:59.050761 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1453/2882] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:17:03.052092 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.052781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.052781 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.082072 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] MFA Rules not processed for user `cc228ac26c454318b15e1a2f198ebdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:17:03.083288 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:17:03.150591 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1430/2883] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:17:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:17:03.154963 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:03.165893 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73586-f65c-42d7-bfe8-5d0893320096 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:03.170749 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:03.171354 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.171687 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.171962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.172592 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:03.174486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:03.202995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1454/2884] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 9 07:17:03 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 07:17:03.206748 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:03.218807 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440475d-7a82-402c-ac28-292a239d4c66 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:03.276782 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:03.277315 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.277594 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.277872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.278489 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:03.280467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:03.296847 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:17:03.303284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1431/2885] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 9 07:17:03 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 07:17:03.307216 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:03.317537 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f03f5-5d14-40dc-806f-9e8b02d5ed19 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:03.322341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:03.322938 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.323217 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.323493 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.324220 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:03.325727 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:03.331403 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:17:03.337586 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1455/2886] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 07:17:03.341381 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:03.352225 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436b1c-ab78-4fb1-909d-0fbbf5b3c485 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:03.356851 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:03.357500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.357786 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.358064 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] PATH_INFO: `/v3/projects/168005fef759450db23c7b6088ae452b/users/249cc253bdda45b8ab140eb19dae253e/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.372845 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] RBAC: Authorizing `identity:create_grant(project_id=168005fef759450db23c7b6088ae452b, user_id=249cc253bdda45b8ab140eb19dae253e, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:03.375495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:03.389123 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1432/2887] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:17:03 2022] PUT /identity/v3/projects/168005fef759450db23c7b6088ae452b/users/249cc253bdda45b8ab140eb19dae253e/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 07:17:03.393026 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:03.403756 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ff71e-b86f-40db-9cb9-f1908346641b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:03.467322 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8503abd-1780-424f-8266-fd6cdbad1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:03.467976 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8503abd-1780-424f-8266-fd6cdbad1b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.468267 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8503abd-1780-424f-8266-fd6cdbad1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.468549 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8503abd-1780-424f-8266-fd6cdbad1b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.469197 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8503abd-1780-424f-8266-fd6cdbad1b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:03.470842 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8503abd-1780-424f-8266-fd6cdbad1b60 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:03.480661 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1456/2888] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 9 07:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 07:17:03.484872 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:03.495114 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8602f-9809-4428-86bc-a25644ff425d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:03.499671 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:03.500320 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:03.500596 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:03.500869 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] PATH_INFO: `/v3/projects/168005fef759450db23c7b6088ae452b/users/249cc253bdda45b8ab140eb19dae253e/roles/d58e69f2b822466daca66a9696521a24` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:03.502779 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168005fef759450db23c7b6088ae452b, user_id=249cc253bdda45b8ab140eb19dae253e, role_id=d58e69f2b822466daca66a9696521a24)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:03.506212 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:03.516481 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1433/2889] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 9 07:17:03 2022] PUT /identity/v3/projects/168005fef759450db23c7b6088ae452b/users/249cc253bdda45b8ab140eb19dae253e/roles/d58e69f2b822466daca66a9696521a24 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:17:07.610421 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40e9d3-6d69-4f54-999f-057dc3f05d9c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:07.610883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40e9d3-6d69-4f54-999f-057dc3f05d9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:07.610974 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40e9d3-6d69-4f54-999f-057dc3f05d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:07.653770 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d40e9d3-6d69-4f54-999f-057dc3f05d9c None None] MFA Rules not processed for user `249cc253bdda45b8ab140eb19dae253e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:17:07.655245 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d40e9d3-6d69-4f54-999f-057dc3f05d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:17:07.770721 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1457/2890] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:17:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:17:07.786744 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:07.790292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:17:07.893680 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137911f-946f-4114-ac3c-494b4a629ab2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:07.900561 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:07.901343 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:07.901628 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:07.902050 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:07.968018 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:07.969606 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:07.980691 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1434/2891] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:17:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:17:17.764658 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=63 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:17:17.975754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:17:23.376918 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:23.377425 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:23.377527 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:23.397807 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] MFA Rules not processed for user `cc228ac26c454318b15e1a2f198ebdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:17:23.398993 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:17:23.405741 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1458/2892] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:17:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:17:23.718082 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:23.773393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cbd02-f358-45ec-86a7-e3b6edbb02c2 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:23.777005 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:23.777605 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:23.777885 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:23.778159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:23.780024 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:17:23.789730 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:23.791136 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:23.799047 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1435/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:17:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:17:33.391663 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:17:33.795416 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=26 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:17:41.124125 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:17:41.135152 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cbc45-accf-498a-afc4-bba40498e748 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:17:41.138621 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:17:41.139213 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:17:41.139488 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:17:41.139795 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:17:41.148928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:17:41.150341 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:17:41.164983 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1459/2894] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:17:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:17:51.159514 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:18:55.178844 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:18:55.180393 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:18:55.190944 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25694b4-0fb5-454c-ae15-38f195a23ea2 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:18:55.194520 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:18:55.195105 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:18:55.195379 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:18:55.195684 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:18:55.197322 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:18:55.207003 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:18:55.208436 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:18:55.217365 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1436/2895] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:18:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:19:05.212759 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:19:37.951820 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:19:37.960582 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64427343-e5ae-470c-9416-48bbb0169b19 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:19:37.964207 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:19:37.964798 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:19:37.965077 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:19:37.965386 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:19:37.967037 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:19:37.980784 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:19:37.982196 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa52721-00c8-468f-b645-3355cf6ce100 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:19:37.993774 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1460/2896] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:19:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:19:39.179395 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:19:39.188415 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa473940-8cbe-46bb-8611-ab592a234ab4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:19:39.192244 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:19:39.192894 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:19:39.193177 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:19:39.193449 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:19:39.195188 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:19:39.209178 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:19:39.210611 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13d25f-fda5-4d0e-8f33-d241d689b966 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:19:39.222622 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1437/2897] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:19:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:19:41.503124 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:19:41.503590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:19:41.503717 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:19:41.522835 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] MFA Rules not processed for user `cc228ac26c454318b15e1a2f198ebdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:19:41.524048 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:19:41.531318 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1461/2898] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:19:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:19:49.216690 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:19:51.518171 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:21:33.773920 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:21:33.774459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:21:33.774459 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:21:33.802532 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] MFA Rules not processed for user `cc228ac26c454318b15e1a2f198ebdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 07:21:33.803767 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:33.827582 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1438/2899] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 9 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 07:21:34.574910 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:21:34.576486 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:34.724343 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1d13-3ef5-4197-a840-3e27a4bada6c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:21:34.729410 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:21:34.730011 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:21:34.730296 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:21:34.730569 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:21:34.732283 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:34.744496 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:21:34.745995 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:21:34.756304 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1462/2900] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:21:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:21:43.825630 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=15 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:21:44.751545 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=63 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:21:47.744492 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:21:47.746077 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:47.769914 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac2c1-337d-48e2-bb45-4b3da90c86c4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:21:47.775495 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:21:47.776126 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:21:47.776420 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:21:47.776674 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:21:47.778519 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:47.791909 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:21:47.793512 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:21:47.805285 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1439/2901] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:21:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:21:51.528624 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:21:51.540857 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236fd1e-5144-41ae-8b76-8e69b8bb2f86 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:21:51.546208 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:21:51.546804 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:21:51.547080 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:21:51.547347 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:21:51.549092 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:51.561662 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:21:51.563229 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:21:51.575337 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1463/2902] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:21:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:21:54.457531 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:21:54.469394 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dffaf-4567-45ad-a508-ffbdfd20f768 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:21:54.474565 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:21:54.475159 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:21:54.475439 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:21:54.475754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:21:54.477484 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:21:54.488769 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:21:54.490476 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:21:54.500187 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1440/2903] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:21:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:22:01.569987 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:22:04.495972 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=15 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:23:18.417796 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:18.437198 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f1f88-1617-4df1-9e2f-5d189c6653f8 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:18.440737 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:18.441323 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:18.441599 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:18.441871 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:18.450889 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:18.452328 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:18.461157 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1464/2904] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:23:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:23:21.697663 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:21.724259 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3ff60-883b-408f-b196-73d661181026 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:21.730486 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80a908c1da74b3fb3e82c8277556e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:21.731127 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:21.731407 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:21.731709 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:21.733559 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:23:21.760483 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:21.762058 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:21.773547 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1441/2905] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:23:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 07:23:28.457269 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:23:31.750094 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:31.760059 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce970d41-a0d9-44d5-a0ee-dfbaee98b5b3 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:31.763731 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '476a1946148e4758b071e9b6ce788bcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:31.764336 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:31.764618 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:31.764888 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:31.768174 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=15 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:23:31.834375 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:31.835839 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:31.856032 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1465/2906] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:23:34.880595 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:34.897745 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714d1ca-39a8-41c5-8474-4c0c06e1484d service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:34.904192 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:34.904827 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:34.905110 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:34.905441 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:34.907479 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:23:34.924388 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:34.926877 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:34.945144 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1442/2907] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:23:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:23:37.934654 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:37.945310 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854151-dce9-493b-b852-9af6ee71c007 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:37.950590 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:37.951240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:37.951525 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:37.951851 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] PATH_INFO: `/v3/users/cc228ac26c454318b15e1a2f198ebdf4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:37.952998 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc228ac26c454318b15e1a2f198ebdf4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:37.955043 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:38.009884 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc228ac26c454318b15e1a2f198ebdf4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.017166 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc228ac26c454318b15e1a2f198ebdf4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.025031 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc228ac26c454318b15e1a2f198ebdf4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.025977 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc228ac26c454318b15e1a2f198ebdf4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.038811 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc228ac26c454318b15e1a2f198ebdf4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.045233 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1466/2908] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:23:37 2022] DELETE /identity/v3/users/cc228ac26c454318b15e1a2f198ebdf4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:23:38.226015 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:38.239011 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91df74-a369-408b-b8e6-21fd50c631a0 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:38.297754 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:38.298387 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:38.298669 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:38.298948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] PATH_INFO: `/v3/projects/ac9f07b87ad643b58d21be9042b64c87` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:38.300168 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9f07b87ad643b58d21be9042b64c87)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:38.302101 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:38.347798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9f07b87ad643b58d21be9042b64c87', 'request_id': 'req-20ad2f42-753d-4af1-90b4-59bc312fc3a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.353798 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9f07b87ad643b58d21be9042b64c87', 'request_id': 'req-20ad2f42-753d-4af1-90b4-59bc312fc3a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.359891 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9f07b87ad643b58d21be9042b64c87', 'request_id': 'req-20ad2f42-753d-4af1-90b4-59bc312fc3a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.386034 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] The token cache is being invalidate because project ac9f07b87ad643b58d21be9042b64c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:23:38.386325 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.386959 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.388467 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1443/2909] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:23:38 2022] DELETE /identity/v3/projects/ac9f07b87ad643b58d21be9042b64c87 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:23:38.392584 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:38.393996 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:23:38.407565 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe96fdd-8d29-4694-be09-b85952b173dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:38.452587 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:23:38.466601 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:38.467240 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:38.467520 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:38.467828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] PATH_INFO: `/v3/users/249cc253bdda45b8ab140eb19dae253e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:38.468962 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] RBAC: Authorizing `identity:delete_user(user_id=249cc253bdda45b8ab140eb19dae253e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:38.470908 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:38.516728 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '249cc253bdda45b8ab140eb19dae253e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.524446 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '249cc253bdda45b8ab140eb19dae253e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.531452 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '249cc253bdda45b8ab140eb19dae253e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.532119 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '249cc253bdda45b8ab140eb19dae253e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.544431 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '249cc253bdda45b8ab140eb19dae253e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.550192 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1467/2910] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 9 07:23:38 2022] DELETE /identity/v3/users/249cc253bdda45b8ab140eb19dae253e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:23:38.719292 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:23:38.720906 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:23:38.737164 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad2f42-753d-4af1-90b4-59bc312fc3a1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:23:38.802045 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b26927358e664fe3981d02119bbe804c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489e2c3ffdae42c6a83d2b2655912ea9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:23:38.802693 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:23:38.803012 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:23:38.803284 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] PATH_INFO: `/v3/projects/168005fef759450db23c7b6088ae452b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:23:38.804564 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=168005fef759450db23c7b6088ae452b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:23:38.806477 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:23:38.815254 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:23:38.866633 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168005fef759450db23c7b6088ae452b', 'request_id': 'req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.872931 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168005fef759450db23c7b6088ae452b', 'request_id': 'req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.879268 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168005fef759450db23c7b6088ae452b', 'request_id': 'req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.892525 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] The token cache is being invalidate because project 168005fef759450db23c7b6088ae452b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 07:23:38.892815 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.893500 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d76f55e1-6bcc-48e7-8a69-fbf095ae3188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 07:23:38.894948 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1444/2911] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 9 07:23:38 2022] DELETE /identity/v3/projects/168005fef759450db23c7b6088ae452b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 07:23:48.546735 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:23:48.889341 dv-cinder-852514-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 07:26:57.384619 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 07:26:57.386295 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:26:57.454966 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b97b0-cb66-4b73-b427-92c13902847b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 07:26:57.512845 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86a917f8f79a494bbd8c18fbefe695ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2051c000cdba46208ac96365247f6b94', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 07:26:57.513447 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 07:26:57.513844 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 07:26:57.514128 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 07:26:57.515928 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 07:26:57.526828 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 07:26:57.528423 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4f05e-ab24-42a1-a89d-71dfbd8b3ab6 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 07:26:57.538883 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1468/2912] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 9 07:26:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 07:27:07.534707 dv-cinder-852514-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=58 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}